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Abstract 

This work consists two parts. In the first part, we completely 
study the heat equation method of Menikoff-Sjostrand and apply 
it to the Kohn Laplacian defined on a compact orientable con- 
nected CR manifold. We then get the full asymptotic expansion 
of the Szego projection for (0, q) forms when the Levi form is non- 
degenerate. This generalizes a result of Boutet de Monvel and 
Sjostrand for (0,0) forms. Our main tool is Fourier integral op- 
erators with complex valued phase functions of Melin and Sjos- 
trand. 

In the second part, we obtain the full asymptotic expansion 
of the Bergman projection for (0, q) forms when the Levi form is 
non-degenerate. This also generalizes a result of Boutet de Mon- 
vel and Sjostrand for (0,0) forms. We introduce a new operator 
analogous to the Kohn Laplacian defined on the boundary of a 
domain and we apply the heat equation method of Menikoff and 
Sjostrand to this operator. We obtain a description of a new Szego 
projection up to smoothing operators. Finally, by using the Pois- 
son operator, we get our main result. 

Resume 

Ce travail est constituee de deux parties. Dans la premiere 
partie, nous appliquons la methode de Menikoff-Sjostrand au 
laplacien de Kohn, defini sur une variete CR compacte orien- 
tee connexe et nous obtenons un deVeloppement asymptotique 
complet du projecteur de Szego pour les {0,q] formes quand la 
forme de Levi est non-degeneree. Cela generalise un resultat de 
Boutet de Monvel et Sjostrand pour les (0,0) formes. Nous util- 
isons des operateurs integraux de Fourier a phases complexes de 
Melin et Sjostrand. 
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Dans la deuxieme partie, nous obtenons un developpement 
asymptotique de la singularite du noyau de Bergman pour les 
{0,q) formes quand la forme de Levi est non-degeneree. Cela 
generalise un resultat de Boutet de Monvel et Sjostrand pour les 
(0, 0) formes. Nous introduisons un nouvel operateur analogue 
au laplacien de Kohn, defini sur le bord du domaine et nous y ap- 
pliquons la methode de Menikoff-Sjostrand. Cela donne une de- 
scription modulo des opereateurs regularisants d'un nouvel pro- 
jecteur de Szego. Finalement, en utilisantl'operateur de Poisson, 
nous obtenons notre resultat principal. 
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Introduction 



The Bergman and Szego projections are classical subjects in several com- 
plex variables and complex geometry. By Kohn's regularity theorem for the 
5 -Neumann problem (1963, [26]), the boundary behavior of the Bergman 
kernel is highly dependent on the Levi curvature of the boundary The 
study of the boundary behavior of the Bergman kernel on domains with 
positive Levi curvature (strictly pseudoconvex domains) became an impor- 
tant topic in the field then. In 1965, L. Hormander (HT)) determined the 
boundary behavior of the Bergman kernel. C. Fefferman (1974, [13]) es- 
tablished an asymptotic expansion at the diagonal of the Bergman kernel. 
More complete asymptotics of the Bergman kernel was obtained by Boutet 
de Monvel and Sjostrand (1976, [11]). They also established an asymptotic 
expansion of the Szego kernel on strongly pseudoconvex boundaries. All 
these developments concerned pseudoconvex domains. For the nonpseu- 
doconvex domain, there are few results. R. Beals and P. Greiner (1988, H]) 
proved that the Szego projection is a Heisenberg pseudodifferential oper- 
ator, under certain Levi curvature assumptions. Hormander (2004, pS]) 
determined the boundary behavior of the Bergman kernel when the Levi 
form is negative definite by computing the leading term of the Bergman 
kernel on a spherical shell in C". 

Other developments recently concerned the Bergman kernel for a high 
power of a holomorphic line bundle. D. Catlin (1997, [5]) and S. Zelditch 
(1998, l34l) adapted a result of Boutet de Monvel-Sjostrand for the asymp- 
totics of the Szego kernel on a strictly pseudoconvex boundary to establish 
the complete asymptotic expansion of the Bergman kernel for a high power 
of a holomorphic line bundle with positive curvature. Recently, a new proof 
of the existence of the complete asymptotic expansion was obtained by B. 
Berndtsson, R. Berman and J. Sjostrand (2004, |i3j). Without the positive 
curvature assumption, R. Berman and J. Sjostrand (2005, f2]) obtained a 
full asymptotic expansion of the Bergman kernel for a high power of a line 
bundle when the curvature is non- degenerate. The approach of Berman 
and Sjostrand builds on the heat equation method of Menikoff- Sjostrand 
(1978, l30l). The expansion was obtained independently by X. Ma and G. 
Marinescu (2006, US]) (without a phase function) by using a spectral gap 
estimate for the Hodge Laplacian. 

Recently, Hormander (2004, [23]) studied the Bergman projection for 
(0,^7) forms. In that paper (page 1306), Hormander suggested: "A careful! 



5 



microlocal analysis along the lines of Boutet de Monvel-Sjostrand should 
give the asymptotic expansion of the Bergman projection for (0, q) forms 
when the Levi form is non-degenerate." 

The main goal for this work is to achieve Hormander's wish-more pre- 
cisely, to obtain an asymptotic expansion of the Bergman projection for 
(0, q) forms. The first step of my research is to establish an asymptotic ex- 
pansion of the Szego projection for (0, q) forms. Then, find a suitable oper- 
ator defined on the boundary of domain which plays the same role as the 
Kohn Laplacian in the approach of Boutet de Monvel-Sjostrand. 

This work consists two parts. In the first paper, we completely study 
the heat equation method of Menikoff-Sjostrand and apply it to the Kohn 
Laplacian defined on a compact orientable connected CR manifold. We 
then get the full asymptotic expansion of the Szego projection for {0,q) 
forms when the Levi form is non-degenerate. We also compute the lead- 
ing term of the Szego projection. 

In the second paper, we introduce a new operator analogous to the Kohn 
Laplacian defined on the boundary of a domain and we apply the method 
of Menikoff-Sjostrand to this operator. We obtain a description of a new 
Szego projection up to smoothing operators. Finally, by using the Poisson 
operator, we get the full asymptotic expansion of the Bergman projection 
for (0, q) forms when the Levi form is non- degenerate. 

We recall briefly some microlocal analysis that we used in this work in 
Appendix A and B. These two papers can be read independently. We hope 
that this work can serve as an introduction to certain microlocal techniques 
with applications to complex geometry and CR geometry. 

Acknowledgements. This work is part of a Ph.D thesis at Ecole Polytech- 
nique in 2008. The author would like to thank his advisor Johannes Sjos- 
trand for his patience, guidance and inspiration. Furthermore, the auther 
is grateful to Bo Berndtsson, Louis Boutet de Monvel and Xiaonan Ma for 
comments and useful suggestions on an early draft of the manuscript. 



6 



Parti 



On the singularities of the Szego 
projection for (0, q) forms 
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1 Introduction and statement of the main results 



Let [X, " T{X)] be a compact orientable connected CR manifold of dimen- 
sion 2n — l, n>2, (see Definition l2.1l l and take a smooth Hermitian metric 
( I ) on CTiX) so that Ai-° r(X) is orthogonal to A^-i r(X) and (u | i;) is real if 
u, V are real tangent vectors, where A^'^ r(X) = A^^°f{X) and CT{X) is the 
complexified tangent bundle. For p & X, let Lp be the Levi form of X at 
p. (See I IL4I I and Definition I2.3I ) Given q, < q < n — I, the Levi form is 
said to satisfty condition Y{q] at p e X if for any \J\ = q, J = [jijz, ■ ■ -Jq), 
l<ji<j2<---<jq<n-l,we have 



where Xj, j = l,...,[n- 1), are the eigenvalues of Lp. (For the precise mean- 
ing of the eigenvalues of the Levi form, see Definition |231) If the Levi form 
is non- degenerate at p, then Y{q] holds at p if and only if ^7 / n-,n+, 
where n+) is the signature of Lp, i.e. the number of negative eigen- 
values of Lp is ri- and n+ + n_ = n - L Let be the Kohn Laplacian on 
X (see Chen- Shaw [6J or section 2) and let D^'^' denote the restriction to 
(0, q) forms. When condition Y{q) holds, Kohn's estimates give the hy- 
poellipicity with loss of one dervative for the solutions of D^f^u = /. (See 
FoUand-Kohn HU, [6] and section 3.) The Szego projection is the orthogo- 
nal projection onto the kernel of D^jf ^ in the space. When condition Y{q) 
fails, one is interested in the Szego projection on the level of (0, q) forms. 
Beals and Greiner (see [1]) proved that the Szego projection is a Heisenberg 
pseudodifferential operator. Boutet de Monvel and Sjostrand (see [llj) ob- 
tained the full asymptotic expansion for the Szego projection in the case of 
functions. We have been influenced by these works. The main inspiration 
for the present paper comes from Herman and Sjostrand |2J. 

We now start to formulate the main results. First, we introduce some 
standard notations. Let be a paracompact manifold equipped with 
a smooth density of integration. We let r(J7) and T*{p.) denote the tan- 
gent bundle of J7 and the cotangent bundle of ft respectively. The com- 
plexified tangent bundle of fl and the complexified cotangent bundle of J7 
will be denoted by Cr(Jl) and CT*{fl) respectively. We write ( ,) to denote 
the pointwise duality between T{iY} and T*[iY}. We extend ( ,) bilinearly to 
C Tin) X C T*{n). Let £■ be a vector bundle over n. The fiber of £" at x e r2 
will be denoted by E^. Let Y cc ft be an open set. From now on, the spaces 



n-l 




(1.1) 
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of smooth sections of E over Y and distribution sections of E over Y will 
be denoted by C°°(y; E] and E) respectively. Let S'[Y; E) be the sub- 
space of E) whose elements have compact support in Y. For 5 e M, 
we let H'^{Y; E) denote the Sobolev space of order 5 of sections of E over Y. 

The Hermitian metric ( | ) on CT{X) induces, by duality, a Hermitian 
metric on CT*{X) that we shall also denote by ( | ) (see lH^). Let A°'^ T*[X) 
be the bundle of (0, q) forms of X. (See I I2.5I I.) The Hermitian metric ( | ) on 
C T*{X) induces a Hermitian metric on A" '? T*{X) (see II2.3I) 1 also denoted by 
(I)- 

We take [dm) as the induced volume form on X. Let ( | ) be the inner 
product on C^CX; A*'-'? T*{X)) defined by 

(/ I g) = I I g(z))(dm), /, g e C^CX; A°'^ r(X)). (1.2) 

Let 

Ti^i^ : L2(X; a"''? T{X)) Keruf 

be the Szego projection, i.e. the orthogonal projection onto the kernel of 
Let 

K^i,){x,y) e X X; if (A°'^ r(X), A"''? r(X))) 

be the distribution kernel of tt^'?' with respect to the induced volume form 
[dm]. Here if (AO'^r*(X),AO''?r*(X)) is the vector bundle with fiber over 
(x,y ) consisting of the linear maps from A" '? T*[X) to A"''? T*{X). 

We pause and recall a general fact of distribution theory. Let E and F 
be vector bundles over a paracompact manifold equipped with a 
smooth density of integration. If A : C^(r2; E] — > @'(r2; F) is continuous, we 
write KA{x,y] or A(x, y) to denote the distribution kernel of A. The follow- 
ing two statements are equivalent 

(a) A is continuous: S'in; E] C°°(J^; F), 

(b) KAec^inxn;jeiEy,F,]). 

If A satisfies (a) or (b), we say that A is smoothing. Let B : C^iil; E) 
®'(r2; F). We write A = BifA—Bisa. smoothing operator. A is smoothing if 
and only if A : H'^^^ [n; E) Hf+^(ri; F] is continuous, for all AT > 0, 5 e M, 
where 

H'^^^in- F) = {ue Flifue H'in- F); V(/? G Co~(ri)} 
and H'^^^^in; E) = H^^^iri; E)f]<g'[n; E). (See Hormander 1221 .1 
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Let Ai'O T*iX) denote the bundle with fiber A^'°T*{X) := A" ! T*iX) at z e 
X. Locally we can choose an orthonormal frame coi(z), . . . , cOn-iiz] for the 
bundle A^''^T*{X), then cui(z),...,a)„_i(2) is an orthonormal frame for the 
bundle A^-^ T*[X]. The real [2n - 2) form co = i"~^coi A cJi A • • • A cy„_i A 6J„_i 
is independent of the choice of the orthonormal frame. Thus co is globally 
defined. Locally there is a real 1 form ojo{z) of length one which is orthogo- 
nal to Ai'° T*{X) © A°'^ T*{X]. ojo[z) is unique up to the choice of sign. Since 
X is orientable, there is a nowhere vanishing (2n — 1) form Q on X. Thus, 
coo can be specified uniquely by requiring that 

coAcoo = fQ, (1.3) 

where / is a positive function. Therefore coq, so chosen, is globally defined. 
We call coo the uniquely determined global real 1 form. 

We recall that the Levi form Lp, p G X, is the Hermitian quadratic form 
on A^ ° Tp{X) defined as follows: 

For any Z, W e A^'O Tp{X), pickZ, W e C^iX; A^'° r(X)) that satisfy 

_ _ 1 / ^ ^ \ (1.4) 

Zip] = Z, W{p) = W. Then Lp{Z, W) = — (^[Z, W]ip) , cooip)) . 

In this work, we assume that 

Assumption 1.1. The Levi form is non-degenerate at each point of X. 
The characteristic manifold of D^f ^ is given by E = E+ (J E", where 

E+ = {(X, O e r (X) \ 0; ? = Awo(x), A > 0} , 

E- = {(X, ^) e r (X) \ 0; ? = Awo(x), A < 0} . (1.5) 

Let (n_, + n+ = n - 1, be the signature of the Levi form. We define 

E = E+ if n+ = q ^ U-, 
E = E~ if n- = q ^ n+, 
E = E"'"|^E~ if n+ — q — n-. 

Recall that (see HU, ^)ifq^ {u-, n+}, then 

KMx,y) e C°°iX X X; if (A°''? r(X), A°''? r(X))). 

The main result of this work is the following 
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Theorem 1.2. LetiX,A^-° T[X]) be a compact orientable connected CR man- 
ifold of dimension 2n — \,n>2, with a Hermitian metric ( | ). (See Deflni- 
tion \2.1\ and Definition \2.2[ ) Let{n-, n+], n- + n+ — n — l,be the signature of 
the Levi form and letq = n- or n+. Suppose n^^^ has closed range. Then, 

is continuous, for all seR, andWF'[K„(q-i) = diag(E x E), whereWF'[K,^[q}) — 
{ix,^,y,ri)eT*iX) x {x,^,y,-ri)eWFiK„i,))}. Here WF {K^i.j] is the 

wave front set ofK„{q) in the sense ofHormander /78|/ . (See also chapter 8 
of 122^.) Moreover, we have 

K^{q) — K^{q) if n+ — q^n-, 

K^w)^K^(q) if n-. = q^n+, 

Kj^(q) = K^(q) + K^{q) H n+ = q = n-, 

where K^(q) {x,y) satisfies 



^^,„(x,y) = J e"t>-'''-y^'sAx,y,t)dt 

with 

sAx,y, t)&Sl-\X X Xx]0,oo[; ^(A°'^r(X),A°'^r(X))), 

oo 

s+{x,y,t)^^sl{x,y)t"-^-i 

]=0 



mS;'_oH^x^x]0,oo[;^(AO'^r*(X),AO.'?r;(X))), where SI'o, m gR, istheHor- 
mander symbol space 118^ (see also chapter 1 of Grigis-Sjostrand IWj), 

sl{x,y) e C^iX X X; if (A"''? r(X), A"''? r{X]]], j = 0,l,..., 

and 

(p+ix,y)eC'^{XxX), Im(/)+(x,y) > 0, (1.6) 

4i+{x,x] = 0, 4)+{x,y]^0 ifx^y, (1.7) 

dxCj)+^0, dyCjj+^O where lmcl)+ = 0, (1.8) 

d:,(l)+{x,y)U=y^a)o{x], d^0+(x,y)U=y = -Wo(x), (1.9) 
(l)+ix,y) = -(j)+{y,x]. (1.10) 

Similarly, 

J> oo 
e'^-i--y^'s.{x,y,t]dt 
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with 

oo 

s.{x,y,t)^Yj^Ux,y)t"-'-} 

in S1-\X xXx]0,oo[; if (A"-'? T*{X], A"-'? T*iX))), where 

sUx,y) e C^iX X X; if (A"-'? r(X), A"-'? r(X))), 7 = 0, 1, ... , 

and when q = n^ = n+, 0_(x,y) = -0_,_(x,y). 

A formula for s'^{x,x) will be given in Proposition \1.8[ More properties of 
the phase cj)+{x,y) will be given in Theorem lLB and Remarl dLB below. 

Remark 1.3. If Y{q - 1) and Yiq + 1) hold then D^^^ has closed range. (See 
section 7 for a review and references.) 

Remark 1.4. If {X,A^'°T{X)) is non-orientable, we also have generalization 
of Theorem ll.21 (See section 10.) 

The phase 0+(x,y) is not unique. We can replace (p+[x,y] by 

$ix,y) = fix,y](j)+ix,y), (1.11) 

where f{x,y] e C°°{X x X) is real and f{x,x) = I, f{x,y) = f{y,x). Then 
(p satisfies HI. 61 1-1 11. lOl l. We work with local coordinates x = (xi,...,X2„-i) 
defined on an open set 57 c X. Let p e Jl. We can check that 

{$"{p, p]U, V) = (((/)+)>, p)U, V) + {dfip, pi U) {dcj)+{p, pi V) 
+ {dfip, pi V) {d(j)+[p, pl U), U,V ecnxi 



where {(p+Y' 



and similarly for (p". The Hessian {(p+Y' 



of cp+ at [p, p) is well-defined on the space 

7(p,p)H+ := {w&CTp{X) x CTp[Xl {d(P+ip,pl W)=o}. 

In section 8, we will define T(p^p)H+ as the tangent space of the formal hy- 
persurface H+ (see 1 18.451 1) at {p, p)eXxX. 

We define the tangential Hessian of (p+{x,y) at {p, p) as the bilinear map: 

T(p,p)H+ X T[p^p)H+ C, 

[U, V) ^ {{(Pltp, p)U, V), U,Ve \,p)H+. 

In section 9, we compute the tangential Hessian of (p+{x,y) at {p, p) 
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Theorem 1.5. For a given point p&X, let Ui{x), [7„_i(x) be an orthonor- 
mal frame of A^'^T^iX) varying smoothly with x in a neighborhood of p, 
for which the Levi form is diagonalized at p. We take local coordinates 
X = [xi,. . .,X2n-i), Zj = Xzj-i + ix2j, j — l,...,n-l, defined on some neigh- 
borhood of p such thatcoo{p) = ^dx2n-\, x(/?) = 0, {-^[p] I ■£^{p)) = 25j^k, 
j, k — l,...,2n — 1 and 

where Cj e C, ^ = 2^9^ ~ j = ^>---'n-l, and j = l,...,n-l, 
are the eigenvalues of Lp. (This is always possible. See page 157 -page 160 
ofUJJ also write y = (yi, . . .,y2n-i], Wj = 3/27-1 + iyzj, ; = 1, . . . , n - 1. 
Then, 

n—l n—1 

(l)+ix,y] = \/2(x2„-i - y2„-i) + 2 1 I \zj -Wjf + '^(^iPijiZjWj - zj Wj) 

7=1 i=i 

+ Cj{ZjX2n-l - Wjy2n-l)+^i^iX2n-l-U>jy2n-l)) + fen-l " y2n-l)/U.y ) 

+ 0[\{x,yfl feC^, /(0,0) = 0, /(x,y) = 7(y,x). (1.12) 

Remark 1.6. With the notations used in Theorem 1 1.51 since fj^^ ^ (0, 0) ^ 0, 
from the Malgrange preparation theorem (see Theorem 7.57 of [22]), we 
have 

(p+ix,y] = gix,y)iV2x2n-i + h{x',y]) 

in some neighborhood of (0, 0), where g, he C°°, g{0, 0) = 1, h[0, 0) = and 
x' = (xi,...,X2„-2). Put 0(x,y) = ^/2x2n~l + h{x',y). From the global the- 
ory of Fourier integral operators (see Theorem 4.2 of Melin-Sjostrand [29\ 
or Proposition IA.20I I, we see that ^+ and (p are equivalent at [p,ojo{p]] in 
the sense of Melin-Sjostrand [29J (see Definition lA. 191 1. Since 0+(x,y) = 
-(p_^{y,x), we can replace the phase (p+{x,y) by 

0(x,y)-^(y,x) 
2 

Then the new function 0+(x,y ) satisfies I I1.12I I with / = 0. 

We have the following corollary of Theorem |1.2[ (See section 9.) 

Corollary 1.7. There exist 

F+, G+, F_, e C°°(X x X; ^(A°'^ T* (X), A°'^ T*iX))) 
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such that 



K^i,, = F+{-i{(l)+{x,y]+mr"+G+logi-ii(j)+ix,y) + iO]], 
K^i^ = F.{-i{(j).[x,y)+mr"+GAog{-i{(t)-{x,y) + m). 



Moreover, we have 



n-l 



F+ = Y,{n - 1 - k)\sl{x,y){-i(i>+{x,y)f+U[x,y)[(l)Ax,y)T, 




n-l 



F.^YSn - 1 - k)\s't{x,y){-i(t>.{x,y)f+f.{x,y){(l,.{x,y)T, 








(1.13) 



where f+[x,y),f-[x,y) e C°°(X x X; ^(A"-'? r*(X), A"-'? r*(X))). 

If we AO'i T*iX), let w^'* : A^'-J+i r*(X) ^ AO'"? r*(X), ^ > 0, be the adjoint 
of left exterior multiplication w^ : A" '? T*iX) A^'^+i r*(X). That is, 



for all u e A"-'? T*iX), v e A°-''+^ T*iX]. Notice that w^'* depends anti-linearly 
on w. 

In section 9, we compute F+(x,x) 

Proposition 1.8. For a given point p & X, let Ui[x),...,Un-i{x) be an or- 
thonormal frame ofA^-°Tx{X), for which the Levi form is diagonalized at p. 
Letej[x), j = I,... ,n—l, denote the basis o/A°'^ T*{X), which is dual to Uj{x), 
j = l,...,n — l. LetXjix), j = \,...,n — \,be the eigenvalues of the Levi form 
Ljc. We assume thatq = n+ and thatXj{p) >Oifl<j< n+. Then 



In the rest of this section, we will give the outline of the proof of Theo- 
rem[r2l Let M be an open set in E" and let /, g e C°°(M). We write f^g 
if for every compact set c M there is a constant Cjf > such that / < Cxg 
and g < Cgf on K. 

We will use the heat equation method. We work with some real local co- 
ordinates X — (xi, . . .,X2n-i) defined on an open set c X. Let [n-, n+), n_ + 



[w^u \v) = iu\ w^'*v), 



(1.14) 
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n+ = n — 1, be the signature of the Levi form. We will say that a e C°°(E+ x 
Jl X ]R2n-i-] jg quasi-homogeneous of degree j if a{t,x,Xr]) = Xi a{Xt ,x,rj) 
for cill A > 0. We consider the problem 



(1.15) 



{dt + U]llu[t,x)^{) inE+xfi 



I m(0,x)= v[x) 

We shall start by making only a formal construction. We look for an approx- 
imate solution of m.lSI I of the form u{t,x) — A{t)v{x), 

A{t)v{x) = -^^;^^^e'^^'^''''^Hy^^^^ (1-16) 



where formally 



a[t,x,r])--'y^aj{t,x,r]), 

7=0 



aj{t,x, rj) is a matrix-valued quasi-homogeneous function of degree —j . 
We let the full symbol of be: 

2 

full symbol of df = ^ Pj{x, 

where Pj{x, ^) is positively homogeneous of order 2 - ; in the sense that 

Pj{x,Xr]) = }}~~' Pj{x,r]), |}7| > 1, A> 1. 

We apply dt + D^'^' formally inside the integral in 1 11.161 1 and then introduce 
the asymptotic expansion of \lt^\ae^^'). Set [dt + \d^\ae^^) ~ and re- 
group the terms according to the degree of quasi-homogeneity. The phase 
ip{t,x,r]) should solve 

^-ipo{x,ilj'J = 0[\lmxpriyN>0 
at ^ (1.17) 

This equation can be solved with lmxp{t,x,r]) > and the phase ijj[t,x,ri] 
is quasi-homogeneous of degree 1. Moreover, 



ipit,x,ri]^{x,r]) onS, d^^r^iip - (x,?7)) = 0onS, 

lmiPit,x, J7) ^ (hi ^I^J | )dist((x, j^),T.)f, Ir/I > 1. 
1 + nn n 
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Furthermore, there exists j/'(oo,x, rj] e C°°(r2 x ]R2«-i) with a uniquely deter- 
mined Taylor expansion at each point of I! such that for every compact set 
K crtx ]R2«-i there is a constant Ck>0 such that 

II II 
Im?/'(oo,x,}7)> }7 (dist((x, |— M > 1. 

\ri\ 

If A e C{T*Q \ 0), A > is positively homogeneous of degree 1 and A|e < 
min Aj, Aj > 0, where ±2 Aj are the non-vanishing eigenvalues of the funda- 
mental matrix of then the solution xp{t,x,iq) of I I1.17I I can be chosen so 
that for every compact set c x ]R2«-i and all indices a, jS, y, there is a 
constant Ca,ji,j,K such that 



<Ca,p,r,Ke~^'-'''''^' onE+x^. 



(For the details, see Menikoff-Sj6strandl30l or section 4.) 
We obtain the transport equations 

{I \^ 
Tit,x,ri,dt,dx]ao = Oi\lmip\ ), 
I |JV (l-J-Oj 

Tit,x,ri,dt,dx]aj + ljit,x,ri,ao,...,aj..i] = Oi\lmip\ ), VA/". 

Following the method of Menikoff-Sjostrand ISOJ, we see that we can 
solve m.lSl l. Moreover, aj decay exponetially fast in t when q ^ ri-, n+, 
and has subexponetially growth in general. (Subexponetially growth means 
that Uj satifies I I5.13I I.) We assume that q = n- ox n+. To get further, we 
use a trick from Berman-Sjostrand [SI. We use db^^^ — U^^^^^dj,, db U^^^ = 
n^b~^^dt and get in a formula asymptotic sense 



Put 



We have 



(5, + n?+'')(e''^a)~0, 

The corresponding degrees of d and a are q + l and q - I. We deduce as 
above that d and a decay exponetially fast in t. This also applies to 

df[a e'^')^dbiVae'''') + V(dbae"^') 
= ^fa(e''^a) + 4*(e''^fl). 
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Thus, dt[ae'^) decay exponetially fast in t. Since dtip decay exponetially 
fast in t so does dta. Hence, there exist 

a; (00, X, Y]) e C°^{r{n]; if (A"''? T*in) , A"-'? T*m, ; = 0, 1, . . . , 

positively homogeneous of degree -j such that aj{t,x,rf) converges expo- 
nentially fast to aj[oo,x, rj], for all 7 = 0, 1, . . .. 

Choose X e C^{W-"~^] so that xi^) = 1 when |)7| < 1 and xi^) = when 
|}7| > 2. We formally set 

and 

In section 6, we will show that G is a pseudodifferential operator of order 
-1 type (5, |). In section 7, we will show that S + oG = I, o S = 0. 
From this, it is not difficult to see that n^i^ = S if d^^ has closed range. We 
deduce that tt^'?' is a Fourier integral operator if d^^ has closed range. From 
the global theory of Fourier integral operators (see [29] and section 8), we 
get Theorem |1.2i 

2 Cauchy-Riemann manifolds, d t -Complex and D^, , a 

review 

We will give a brief discussion of the basic elements of CR geometry in a 
setting appropriate for our purpose. General references for this section are 
the books pj, Boggess 14J and |6j. 

Definition 2.1. Let X be a real manifold of dimension 2 n — 1, n > 2, and 

let Ai'O T{X) be a subbundle of Cr(X). The pair (X, A^ o r(X)) is called a CR 
manifold or a CR structure if 

(a) dimcA^'°TpiX) = n-l,peX, 

(b) Ai'O r(X)n A°'^ T{X) = 0, where A"-! T{X) = A^-'^UX), 

(c) For any M, e C~(L/'; Ai'«r(X)), [M, 1^] e C°°([7; Ai'Or(X)), where U is 
any open subset of X. 
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Definition 2.2. Let (X,Ai'T(X)) be a CR manifold. A Hermitian metric 

( I ) on CT{X) is a complex inner product ( | ) on each CTp{X) depending 
smoothly on p with the properties that A^ " Tp{X) is orthogonal to A^'^ Tp{X) 
and [u I v)is real ifu,v are real tangent vectors. 

Until further notice, we assume that [X, A^-" T[X)) is a compact orientable 
connected CR manifold of dimension 2n — l, n>2, and we fix a Hermitian 
metric ( | ) on C T{X). Then there is a real non-vanishing vector field F on X 
which is pointwise orthogonal to A^'" T{X) ® A^-^ T{X). 

The Hermitian metric ( | ) on CT[X) induces, by duality, a Hermitian 
metric on Cr*(X) that we shall also denote by ( | ) in the following way. For 
a given point z e X, let F : CTz{X) CT*[X) be the anti-linear map defined 
by 

iu\v)={u,rv), u,vGCT^iX). (2.1) 
For jU G Cr*(X), we put 

(6j|/i) = (r-Vir"^a>). (2.2) 

Let A'[CT*iX)l r e N, be the vector bundle of r forms of X. That is, the 
fiber of A''(Cr*(X)) at z e X is the vector space A' (C r*(X)) of all finite sums 
of K A • • • A l^, Vj eC T*{X), j = l,...,r. Here A denotes the wedge product. 
The Hermitian metric ( | ) on A'^(Cr*(X)) is defined by 

iUiA---AUr I ViA---AVr) = det[iUj \ Vk))^^j^^^, 

Uj,Vk&Cr{X), i,k = \,...,r, (2.3) 

and we extend the definition to arbitrary forms by sesqui-linearity. 

Let A''(Cr(X)), r G N, be the bundle with fiber K^CT^iX)) at z G X. The 
duality between A''(Cr(X)) and A''(Cr*(X)) is defined by 

{ViA--- AVr,UiA---AUr) = det{^{vj,Uk))^^. 

Uj G Cr*(X), Vj G <CT{X),j = 1, . . . , r, and we extend the definition by bilin- 
earity. 

For z G X, let v &CTz{X). The contraction 

: A''+Hcr(X)) ^ A''(Cr(X)) 

is defined by (i'iA---Ai';-, v-'u) = {v AviA---AVr, u) for all u & AJ'+\€.T*{X)), 
VjeCUXlj = l,...,r. 

We have the pointwise orthogonal decomposition 

c r(x) = A^'° r(x) © A°'i r(x) © cf. (2.4) 
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Define A^'° T*[X) = (A^'i T{X) © C7)^ c CT*iX] and A^'i T*{X) = (Ai-" T{X) © 
CY)^ c Cr*(X). We can check that A^-°T*iX) = TA^'°T,iX), A°-^T*iX) = 
TA°-^T^[X) and Ai or*(X) is pointwise orthogonal to A'''ir*(X), where T is 
as in I I2.1I I. For (7 e N, the bundle of (0, q) forms of X is given by 

A°''?r(X) = A'?(A°'ir(X)). (2.5) 

We use the Hermitian metric ( | ) on A'' '? T*{X), that is naturally obtained 
fromA'?(Cr*(X)). 

Let d : C^iX; A'-{CT*{X]]] C»(X; A'-+i(Cr*(X))) be the usual exterior 
derivative. We recall that the exterior derivative d has the following prop- 
erties, where [b], [c] are special cases of Cartan's formula: ifv<^ = v-'dco + 
div-'co). Here v is a smooth vector field, co is a. q form and J^yCo is the Lie 
derivative of co along v. 

(a) If / e C'^(X) then ( V, df) = V[fl V e C^iX; CT[X)). 

(b) If (/) e C°°(X; CT*iX]] then 

{V,AV2,d(P) = V,[{V2,(t))]-V2i{V,,(j)))- {m,V2],(P), (2.6) 
where K, ^ e C°^(X; Cr(X)). 

(c) If (/) e C°°(X; A'?-i(Cr*(X))), f7 > 2, then 

( K A • • • A y^, ) = - ( 14 A • • • A y^, 

+ K((^^2A---Ay^,(/))) 

-{[V,,V2]AV3A---A%,(j)) 

{V2AVsA---Am,%],(l)), (2.7) 

where Vj e C^CX; Cr(X)), j = l,...,q. 
We have the pointwise orthogonal decomposition: 

c r (X) = Ai'° r(x) © A°'i r (X) © {a^o; a e c} . (2.8) 

We recall that coq is the uniquely determined global real 1 form (see 1 11.311 ). 
We take Y (already introduced after Definition[Z!2land JZil l) so that || F|| = 1, 
{Y,coo) = — I. Therefore Y is uniquely determined. We call Y the uniquely 
determined global real vector field. 

Definition 2.3. For p &X, the Levi form Lp is the Hermitian quadratic form 
on A^'° Tp{X) defined as follows: 

For any Z, W e A^'O Tp[X), pickZ, W e C^iX; A^-° TiX]) that satisfy 

_ _ 1 / ^ ^ \ (2.9) 

Z[p)=Z, W[p] = W. Then Lp(Z, W)^ — (^[Z, W]ip) , cooip)) . 
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Here [Z , VK] = ZW- WZ denotes the commutator of Z and W. 

Recall that Lp does not depend of the choices of Z and W (see page 153 
of m or using fZHl ll 

Lemma 2.4. LetZ, W e C°°iX; Ai-" r(X)). We have 

^ ( [Z , W]ip) , cooip)) = {zip) A Wipldcooip]) . (2.10) 

Definition 2.5. The eigenvalues of the Levi form at p G X are the eigen- 
values of the Hermitian form Lp with respect to the inner product ( | ) on 

A''°TpiX). 

For U,Ve C^iX; A^ " T[X]), from Q, we can check that 
[U ,V]ip] = -i2i)LpiU{p) ,V(^])Y{p)+h, heA'-^TpiX)®A°''TpiX). (2.11) 
Next we recall the tangential Cauchy-Riemann operator d b and db . Let 
^o.ci . j*f^x)) A"'-? T%X) 
be the orthogonal projection map. 

Definition 2.6. The tangential Cauchy-Riemann operator ^j, is defined by 

'db = 71"-'?+^ o d : C°°(X; A"''? T*{X]) C^iX; A°-'^+^ T*[X)). 
The following is well-known (see page 152 of [l] or using I I2.7I I) 

— 2 

Proposition 2.7. We have — 

Let db be the formal adjoint of db, that is [dbf \ h] — if \ db h), where 
/ e C~(X; A"-'? T*iX)), h e C^iX; A°''i+^ T*iX)) and ( | ) is given by lO- 4* is 
a first order differenticil operator and [db Y = 0. The Kohn Laplacian Ub is 
given by 

\Jb = dbdb +db db- 

From now on, we write to denote the restriction to (0, q) forms. 

For Zq e X, we can choose an orthonormal frame ei(z),...,e„_i(z) for 
^0,1 T*[X) varying smoothly with z in a neighborhood of Zq. Let Zj{z), j = 
1, 1, denote the basis of A°'i T^iX), which is dual to ej[z], j = l,...,n- 
1. Let Z* be the formal adjoint of Z^, j = 1, 1. That is, [Zjf \ h) — 
if I Z*h),f,h e C°°(X). We have the following (for a proof, see page 154- 
page 156 of HI) 
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Proposition 2.8. With the notations used before, the Kohn Laplacian is 
given by 

i=i i,k=i 

+ s{Z) + e{Z*) + zero order terms, (2.12) 

where s[Z) denotes remainder terms of the form ^ ak{_z)Zk with ak smooth, 
matrix-valued and similarly for s{Z*) and the map 

is defined by 

{e]eT o [Zj,Zimz)ej, A - A = [Zj,ZlWie^e'^n oej.A-A e^^ 
and we extend the definition by linearity. We recall that e^'* is given by (1.14\) . 

3 The hypoellipicity of Ot 

We work with some real local coordinates x = (xi, . . . , X2n-i) defined on an 
open set ncX.We let the full symbol of df be: 

2 

full symbol of □^^ = ^Pj(x,?) (3.1) 

7=0 

where pj{x,^] is positively homogeneous oforder2-;. Letqj,] = l,...,n- 
l, be the principal symbols ofZj,j — l,...,n-l, where Zj, j = l,...,n-l, 
are as in Proposition l2.8[ Then, 

n-l 

i=i 

The characteristic manifold S of is 

E = {(x, E,) e r (X) \ 0; ? = Acooix), A^O}. (3.3) 



From 1 13.21 1, we see that po vanishes to second order at E. Thus, E is a doubly 
characteristic 
S is given by 



characteristic manifold of The subprincipal symbol of \df^^ at (xo, <f o) e 



Pl{x„ ?o) = Pi(xo, ?o) + ^ y .^"^^"l^"-^ e ^(A°'; r (X), A"-; r (X)). (3.4) 

2^ oXjO^j 
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It is well-known (see page 83 of Hormander ||T9l) that the subprincipal sym- 
bol of is invariantly defined on E. 

For an operator of the form ZJZ, this subprincipal symbol is given by 




and the contribution from the double sum in 1 12.121 1 to the subprincipal 
symbol of [d^^ is 

j,k=i 

where denotes the Poisson bracket of qj and q^.. We recall that 

{^i-^ J = Y.T=~Aw-/^ - SH^- the subprincipal symbol of ajf^ 

on E, 

n—l n—1 ^ 

;=i '^^ j.k=i ' 

From fZTll . we see that [Zk,Zj] = -{2i)L(Zk,Zj)Y+h, h e C°°[X; A^-°TiX)® 
\o,i T{X)]. Note that the principal symbol of is —'q^- Hence, 

{^k' } = {2i)L{Zk,Zj)criY on E, (3.6) 
where (Jiy is the principal symbol of i Y. Thus, 

n-l n-l 

pl = {^L(Zj,Zj)- 2e;e^'*L(Zfc,Z,))cr,y onE. (3.7) 

In the rest of this section, we will assume the reader is familiar with some 
basic notions of symplectic geometry For basic notions and facts of sym- 
plectic geometry, see chapter XVIII of [19| or chapter 3 of Duistermaat HI]. 

From now on, for any / e C°°{T*{X)), we write Hf to denote the Hamil- 
ton field of /. We need the following 

Lemma 3.1. We recall that we work with Assumption ] E is a symplectic 
submanifold ofT*{X). 

Proof. Let p e E. Note that 

E = {(X, Q e r{X) \ 0; qj{x, Q^q^ix, ^) - 0, ; = 1, . . . , n - l} . 

Let Crp(E) and Crp(r*(X)) be the complexifications of TpCE) and Tp{T*{X)) 
respectively. We can choose the basis H^^ Hq^_^ , Hq^ Hq^^^ for TpiX)^, 



22 



where Tp{X)-^ is the orthogonal to C7^(S) in CTp{T*{X)) with respect to the 
canonical two form, 

a = d^Kdx. (3.8) 

In view of I I3.6I I. we have (j[Hc,., H^J = [qjy'q^.^ = -.UZk,Zj')a iy on S. We 
notice that ^qj,qk\^ = on E. Thus, if the Levi form is non-degenerate at 
each point of X, then a is non- degenerate on 7},(E)^, hence also on C7^(E) 
and E is therefore symplectic. □ 

The fundamental matrix of po at p — [p, ^o) e E is the linear map Fp on 
Tp{T*{X]) defined by 

ait,FpS)^{t,p'^ip)s), t,seTpiriX)), (3.9) 

where a is the canoniccil two form (see 1 13.81 1) and 



We can choose the basis Hq^,...,Hq„_i,H-q^,...,Hq„_^ for TpCE)^, where 
Tp{X)^ is the orthogonal to Crp(E) in CTp[T*[X)) with respect to canoni- 
cal two form. We notice that Hp^^ = '^ji^jHqj + qjHq.). We compute the 
linearization of Hp^ at p 



Hpoip + Yj^hHq, + StHq^ )) = 0{\t,sf) + Y,tk{qk, qj } H, 

j.k 

+ ^Sk {qk,qj}Hq^- 



j.k 

So Fp is expressed in the basis Hq„..., ,Hq^,---, ^ by 



Again, from 1 13.611 . we see that the non-vanishing eigenvalues of Fp are 

±2ajCTiY{p), (3.11) 

where Xj, j = l,...,n — \, are the eigenvalues of Lp. 

To compute further, we assume that the Levi form is diagonalized at the 
given point pGX. Then 

Y,^e^e'^-*LpiZk,Zj)aiY^Y,^e';e^-*LpiZj,Zj)aiY. 

j.k j 
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From this, we see that on E and on the space of (0, q) forms, + |tf F has 
the eigenvalues 

n-l 

'^\Aj\\aiY\+'^?^jO-iY-'^^jO-iY, \ J\ = q, 

7=1 j^J jej (3.12) 

J^Ul'j2,---,jq), l<jl<j2<---<jq<n-l, 

where tr F denotes ^ |, i/i^ are the non- vanishing eigenvalues of Fp . 

Let n+), ri- + n+ = n — I, he the signature of the Levi form. Since 
{Y, coo) = -1, we have ctiy > on E+, (7,y < on S". (We recall that E+ and 
E" are given by I I1.5I I.) Let 

inf(/7Q + ^tr ^) = inf |a; A : eigenvalue of + ^tf 
From 1 13.121 1, we see that on E+ 

inf(p^ + -t~rF)j "^^""^ . (3.13) 

2 [ >0, 

On E- 

inf(p^ + -tfF)j . (3.14) 

Let n be an open set in R^. Let P be a classical pseudodifferential op- 
erator on of order m > 1. P is said to be hypoelliptic with loss of one 
derivative if u e S'in] and Pu e H^^^in) implies u e W+^-\n]. 

We recall classical works by Boutet de Monvel [9] and Sjostrand 1321 . 

Proposition 3.2. Let ft be an open set in M^. LetP be a classical pseudodif- 
ferential operator on fL of order m> I. The symbol ofP takes the form 

(Jp{x, E,) ~ p,n{X, + Pm-l{X, + Pm~2{X, ?) + •••, 

where pj is positively homogeneous of degree j . We assume thatH — P'^W 
is a symplectic submanifold of codimension 2d, Pm^O and pm vanishes to 
precisely second order on E. Let F be the fundamental matrix ofpm- Let 
pt, be the subprincipal symbol of P. Then P is hypoelliptic with loss of one 
derivative ifand only if pl^{p)+'Y^.^^{^+ a j)\pj\ at every point p gE for 
a// («! , a2 , . . . , ffd ) e N'' , where ± ijjij are the eigenvalues ofF at p . 

Proposition |3.2| also holds if P is a matrix- valued classical pseudodiffer- 
ential operator on of order m > 1 with scalar principal symbol. 
From II3.13I I. J3.14I I and Proposition l3.21 we have the following 
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Proposition 3.3. Dj; is hypoelliptic with loss of one derivative if and only if 
Y{q) holds at each point ofX. (We recall that the definition of Y{q) is given 
by (LB-) 

Remarks A. Kohn's estimates give the hypoellipicity with loss of one der- 
vative for the solutions n^^'' u—f under condition Y{q). (See FoUand-Kohn 
p4| .) Kohn's method works as well when the Levi form L is degenerate. 

4 The characteristic equation 

In this section, we consider the characteristic equation for dt + \J^^\ 

Let po{x, ^) be the principal symbol of We work with some real local 
coordinates x = [xi,X2, . . .,X2n-i) defined on an open set ilcX. We identify 
J7 with an open set in M^""^ Let Jl"- be an almost complexification of il. 
That is, is an open set in C^"-! with J^cp|]K2«-i ^ ^ ^Ve identify T*{n) 
with n X ]R2«-i Similarly let T*{n)c be an open set in C^""! x C^""! with 
T*in)c f]{M.^"-'^ X E2«-i) = T*in). In this section, for any function /, we also 
write / to denote an almost analytic extension. (See Definition |A.1[ ) We 
look for solutions ijj[t,x,ri)e C'^{R+ x T*{n] \ 0) of the problem 

_ -^-/po(x,i/';) = 0(|lmj/>|'^), ViV>0, ^^^^ 
^\t=o = {x ,}?) 

with lrail}{t,x,r]) > 0. More precisely, we look for solutions ip{t,x,ri) G 
C°°(l+x r*(J7)\0) with Imj/'(f,x,)7)>0 such that ^^=0 = {x ,ri) and for 
every compact set K c T*{iY} \ 0, AT > 0, there exists Ck,n > 0, such that 

< Ciciv |lm?/)|^ onM+ x K. 

Let fix, a gix,Q e C^iT*in)c). We write / = g mod |Im(x,?)r if, 
given any compact subset K of r*(J7)c and any integer N>0, there is a con- 
stant c> such that |(/- gXx,?)| < c |Im(x, c^)|~, V(x,c^) e K. Let U and 
Vhe C°° complex vector fields on T*{n)c. We write U= V mod |Im(x,cf)r 
if [/(/) = vif) mod |Im(x, and U(f) = v(f) mod |Im(x, ^T, for all al- 
most analytic functions / on r*(J7)c. In Appendix A, we discuss the notions 
of almost analytic vector fields and equivalence of almost analytic vector 
fields. 

In the complex domain, the Hamiltonian field Hp^ is given by Hp^ — 
YeII^^I^- notice that Hp^ depends on the choice of almost analytic 



dip 
Jt 



ipo{x,ip' 
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extension of po but we can give an invariant meaning of the exponential 
map exp[-itHpg), t>0. Note that Hp^ vanishes on E. We consider the real 
vector field -iHpoH — iHpg. Let $(f,p) be the -/HpoH — iHpg flow. We notice 
that for every T > there is an open neighborhood J7 of S in T*(SY}c such 
that for all < t < T, ^{t,p) is well-defined if p e C7. Since we only need 
to consider Taylor expansions at E, for the convenience, we assume that 
$(t,p) is well-defined for all ? > and p e r*(r2)c. The foUoAving follows 
from Proposition lA.121 

Proposition 4.1. Let ^{t,p) be as above. Let U be a real vector field on 
T*{n)c such that U = -iHp^ + -iHp^ mod |Im(x,^)r. Let Ht,p] be the 
U flow. Then, for every compact set K c T*{^)c, N>0, there is c^.Kit) > 0, 
such that 

\Ht,p)-Ht,p]\ < cw,jf(0dist(p,sr, peK. 
For t > 0, let 

Gt = {ipMt,p]y,per[n)c}, (4.2) 

where ^[t,p] is as in Proposistion l4.1[ We call Gt the graph of exp(-/fHpJ. 
Since Hp^ vanishes on E, we have $(f,p) = p if p e E. Gt depends on the 
choice of almost analytic extension of po. Let po be another almost analytic 
extension of po. Let Gt be the graph of exp{-itHpJ. From Proposition l4.1[ 
it follows that Gt coincides to infinite order with Gt on diag(E x E), for all 
f >0. 

In Menikoff-Sjostrand ||30], it was shown that there exist 

git,x, T]], h{t,x, ri) e C'^iR+ x rin)c) 

such that Gf = {{x,g{t,x,ri),h{t,x,ri),riy, [x,ri)e r*(Jl)c}. Moreover, there 
exists xp{t,x,ri)e C°°(E+ x T*{yL)c) such that 

g{t,x,iq)-xp'J^t,x,iq) and h{t ,x,r])- xp'^{t ,x,r]) 

vanish to infinite order on E, for all f > 0. Furthermore, when {t,x,rj) is 
real, ip{t,x, rj) solves 1 14.11 1 and we have, 

lmip{t,x,ri]^-^[disti[x,r]],J:]f, t>0,\ri\ = l. (4.3) 

For the precise meaning of ^, see the discussion after Proposition ! 1.8[ 
Moreover, we have the following 

Proposition 4.2. There exists ip {t ,x,r]) e C'^(R+xT*[n]\0] such thatlmxp > 
with equality precisely on {{0} x T*{n) \ 0)[j{R+ x E) and such that 1 14. ID 
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holds where the error term is uniform on every set of the form [0, T]x K with 
T > and K c T*(SY} \ compact. Furthermore, ip is unique up to a term 
which is 0(| Irru/' I ) locally uniformly for every N and 

ip{t,x,ri]^{x,r]) onS, d^,r^[xp- (x,}7)) = 0onE. 

Moreover, we have 

I I n ■> II 

lmipit,x,ri)^\ri\ 4Sdist((x, f > 0, h > 1. (4.4) 

1 + f m m 

Proposition 4.3. There exists a function ip{oo,x,ri] e C°°(r*(r2) \ 0) with a 
uniquely determined Taylor expansion at each point of S such that 

For every compact set K c r*(Jl) \ there is a constant Cjf > such that 

lm.il)[oo,x,q)>CK\rj\ (dist ((x , -p- , ), Y?)Y , 

\r)\ 

dx,r,{ip{oo,x,ri)- (x,)7)) = 0on E. 

If ?\- & C(r*(r2)\0), A > awd AIj: < min|Aj|, where ±i\Xj\ are the non- 
vanishing eigenvalues of the fundamental matrix of then the solution 
ip{t,x, q) o/ 1 14. Il l can be chosen so that for every compact set K c T*{^) \ 
and all indices a, p,y, there is a constant Ca,p,j,K such that 



d^dPdJ{xl;{t,x,q)-xp{oo,x,q)) 



<Ca,p.r,Ke~^^''''^'^' onn+xK. (4.5) 



For the proofs of Proposition |4.2| and Proposition |4.3[ we refer the reader 
to HSni. From the positively homogeneity of po, it follows that we can 
choose ijj[t,x,ri) in Proposition 14.21 to be quasi-homogeneous of degree 1 
in the sense that x,A}7) = A?/^(Af,x, 17), A > 0. (See Definition lSTTl ) This 
makes ip{oo,x, 17) positively homogeneous of degree 1. 

We recall that po = q{q^ H h qn-iq„_i. We can take an cQmost analytic 

extension of po so that 

Po{x,0 = Poix,a (4.6) 

From 1 14.61 1, we have 

dijj — / I ijv 

-—[t,x,-q)- ipo[x,ip^[t,x,-ri)) = 0{\lmijj\ ], t>0, 

for all N>0, [x, q) real. Since po[x, = po{x, we have 

dih — / I iN 

—^it,x,-q)-ipoix,-ip^{t,x,-q)) = Oi\lmilj\ ],t>0, (4.7) 
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for all N >0,{x,r]) real. From Proposition |4.2i we can take ipit,x, 17) so that 
ip{t,x,ri) = -ip{t,x,—ri], ix,ri) is real. Hence, 

ip{oo,x,ri] = -il!{oo,x,-ri), (x, 17) is real. (4.8) 

Put Gf = |(y,r7,x, (^); ix,^,y,ri) e Gtj, where Gt is defined by 114. 2l i. From 
1 14.61 1. it follows that $(?,p) = $(-f ,p), where $(f,p) is as in Proposition|4.1i 
Thus, for all t > 0, 

Gt^Gt. (4.9) 

Put 

Ct = {(x, ijj'^{t,x, r]l xp'^{t,x, r]l r]]; (x, r]) e r(J7^)} (4.10) 

and Ct = \^[y,r],x,^); {x,^,y,ri] e Cfj. Since Cf coincides to infinite order 
with Gf on diag(I! x E), for all t >0, from 1 14.91 1, it follows that Cf coincides 
to infinite order with Cf on diag(E x S), for all t >0. Letting f ^ 00, we get 
the following 

Proposition 4.4. Let 

Coo = {(x, ip'^ioo,x, T]), ijj'^[oo,x, J7), J7); (x, r]) e T* (J^^)} (4.1 1) 

and Coo = {(y . (x, (^,y, ry) e Coo}- Coo coincides to infinite order 

with Coo on diag(S x E). 

From Proposition l4.4l and the global theory of Fourier integral operators 
(see Proposition |A.20| | , we have the following 

Proposition 4.5. The two phases 

ip{oo,x, T]) - (y, T]) , -?/^(oo,y, r]) + (x, r]) e C°°(J^ x x M^^-i) 

are equivalent in the sense ofMelin-Sjdstrand l2^. (See Definition \A.l^ ) 

We recall that 

S = {(x,<f)er(n)\0;^7,(x,?) = ^/x,?) = 0, 7 = l,...,n-l}. 

For any function / e C°°(r*(r2)), we use / to denote an almost analytic 
extension with respect to the weight function dist((x, (^),E). (See Defini- 
tionlAH) Set 

£ = {(x,^)er(n)c\0;^,(x,?) = f .(x,?) = 0, 7 = 1,..., n - l} . (4.12) 

We say that E is an almost analytic extension with respect to the weight 
function dist((x,^),E) of E. Let /(x,?), g(x,<^) e C^iW), where W is an 
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open set in r*(r2)c. We write f = g mod if, given any compact sub- 
set K of W and any integer N > 0, there is a constant c > suchi thiat 
|(/-g)U,?)| < cdist((x,^),Er, V(x,(f) e iC. From tiie global theory of 
Fourier integral operators (see Proposition |A.20| l , we only need to consider 
Taylor expansions at E. We may work with the following coordinates (for 
the proof, see [SO]) 

Proposition 4.6. Let p e E. Then in some open neighborhood Tofp in 
T*{n)c, there are functions xj e C°°(r), e C^OT], ; = 1, . . . , 2n - 1, such 
that 

(a) Xj,E,j, i = l,...,2n-\, are almost analytic functions with respect to the 
weight function dist((x, E,), E). 

(b) det [f||j] 7^ on (r)R, where (Ffe = T*{n) and x = (xi, . . .,X2n-i), 

? — (?l>---'?2«-l)- 

(c) Xj, E,j, i — l,...,2n- I, form local coordinates of T . 

(d) [x, ^) is symplectic to infinite order on E. That is, {Xj,Xfc} = mod d^, 

= Omod d"^, {E,j,Xk} — 5j k mod d^, where j, k = l,...,2n-\. 
Here{/,g} = f|ff-ffff,/,gGC-(n. 

(e) We write x' , x" , 1' , 1" to denote (xi , . . . , x„ ), (x„+i , . . . , X2„-i ), (l^i , . . . , ^„ ) 
and[E,n+i, ^2n-i) respectively. Then,T.f]r coincides to infinite order 
with {(x, x" = 0, ^" = o} on E flCrV and 

sP|(F)k = {(x, x" = 0, C = 0, x' and f are real } . 

Furthermore, there is a (n — 1) x (n — 1) matrix of almost analytic functions 
Aix, ^) such that for every compact setKcT and N>0, there is a Ck,n > 0, 
such that 



^ Ck,n 



JV 



on K, 



p^{x,l)-i{A{x,l)x",l") 

and when x' and E,' are real, A[^' , 0, E,' , 0) has positive eigenvalues 

l^il , . . . , |A„_i| , 

where ±/Ai, . . .,±.iXn-\ are the non-vanishing eigenvalues of Fix', 0, ^',0), 
the fundamental matrix of In particular, 

^trA[x', 0, r , 0) = ^t'? F(x', 0, f , 0). 
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Formally, we write 

Po[x, = i (a{x, l)x", I") + 0(|(r, roH- (4.13) 

Remark 4.7. Set 

E = {it,x, ^,y, r]]eR+x x r (f7)c; (x, ?,y, r?) e q} , 

where Cf is defined by II4.10I I. Let [x,^) be the coordinates of Proposi- 
tion |4?6l In the work of Menikoff-Sjostrand fSOl, it was shown that there 
exists ipityX, rj) e C°°(]R+ x T), where T is as in Proposition l4.6[ such that 



^ - ipoix, {j^'^) = 0{\{x", ri"flior all AT > 0, 
j/'|t=o = (x 



and ip{t,x,ff) is of the form 

{pit,x, rj) = {x', f?') + 0'^' ?]") + {P2{t,x, rf] + ^^{t,x, ff) + - , 

(4.14) 

where A is as in Proposition |4.6l and xp j{t,x,r]) is di homogeneous poly- 
nomial of degree ; in (x", f]"]. If A e C{T*in) \ 0), A > and Als < min 
where ± i | Xj | are the non- vanishing eigenvalues of the fundamental matrix 
of then for every compact set K c sP|(r)K and all indices a, p,Y, j, 
there is a constant Ca,p,Yj,K such that 



d,-d~^dHiPj{t,x,fi)) 



<Ca.p.r.Ke-'^'-'^^' oni 



x^. 



(4.15) 



Put £ = { ( X , If ( ? , X , f| ( ? , X , rjO, r?); ? e M+ , X , e C-(r)} . We notice 
that E coincides to infinite order with i? on M+ x diag ((S P|(rV) x (S n(r)R))- 
(See Ii30< .) 



5 The heat equation, formal construction 

We work with some real local coordinates x = (xi, . . .,X2n-\) defined on an 
open set J7 c X. We identify T*[n) with ^ x ]R2/i-i_ 

Definition 5.1. We will say that a e C°°(]R+ x r*(r2)) is quasi-homogeneous 
of degree j if a{t,x,Xrj) — Xi a{Xt,x,r]) for all A > 0. 

We can check that if a is quasi-homogeneous of degree j , then dj a 
is quasi-homogeneous of degree ; - )8 + 7. 
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In this section, we consider the problem 

f idt+df)u{t,x] = inE+xJ7 



(5.1) 

[ u[0,x) = v[x) 

We shall start by making only a formal construction. We look for an approx- 
imate solution of 1 15.11 1 of the form u{t,x) = A{t)v{x], 

^itMx) = ^^^^ J J e Wt.x,.)-(y..))^(f ri)v{y)dydr) (5.2) 



where formally 



a{t,x,r])^y^aj[t,x,rfj, 



ajit,x,r]) e C'^{R+ x T*{ny,Se[A°-''T*[n),A°-iT*{n))), aj{t,x,r]) is a quasi- 
homogeneous function of degree -j. 
We let the full symbol of \J^^^ be: 



full symbol of = ^ pjix, 



where Pj{x, ^] is positively homogeneous of order 2- j. We apply dt + nt^^ 
formally under the integral in 1 15.21 1 and then introduce the asymptotic ex- 
pansion of dfiae''!'). (See Proposition EH) Setting [dt + dfXae''^') ~ 
and regrouping the terms according to the degree of quasi-homogeneity. 
We obtain the transport equations 



(5.3) 



I Tit,x,ri,dt,d^)ao = Oi\lmip\'^iyN 

\ Tit,x,ri,dt,d^)aj + Ijit,x,ri,ao,...,aj-i] = 0{\lmilj\ ), VAA. 
Here 

2n—l o o 

Sapo a 
-j^ix,ipj— + qit,x,r]) 

where 

q[t,x,rj)-p,ix,rp^)+^.2^^ 5?,^?. dxjdx, 

and Ij is a linear differential operator acting on ao,ai,... ,aj-i. We note 
that q{t,x, rf) q[oo,x, rj) exponentially fast in the sense of 1 14. 5l l and that 
the same is true for the coefficients of Ij . 
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Let Cf , £■ be as in l lOOl l and Remark|43 We recall that for t > 0, 

i . . dxh dip \ 

[ ox or] ] 

and for t > 0, (CJm = diag(E x E) = {(x, x, E,) e T*in] x T*{n]; (x, e S}. 

If we consider Uq, (ii, . . . as functions on E, then the equations I I5.3I I in- 
volve differentiations along the vector field v — jj — iHp^. We can consider 
only Taylor expansions at S. Until further notice, our computations will 
only be valid to infinite order on S. 

Consider v as a vector field on E. In the coordinates {t,x,ri) we can 
express v: 

o 2n-l o 



We can compute 



div(v) = - 



For a smooth function a{t,x, rf) we introduce the | density on E 

a = a{t,x,r])\/ dtdxdrj 

which is well-defined up to some factor ii^. (See Hormander HTH).) The Lie 
derivative of a along v is Ly{a} — (v(a)+|div(v)a)y^ dtdxdrj. We see from 
the expression for T that 

{Ta)\/ dtdxdrj = {Ly + pl{x,il}'J^t ,x,rj))){a\/ dtdxdrj), (5.5) 

where pl{x,E,) = pi{x,Q + ^Xj=/ "^g^gg'^ ^'^^ subprincipal symbol (see 
I I3.4I I). Now let (x, i^) be the coordinates of Proposition |4.6[ in which po takes 
the form I I4.13I I. In these coordinates we have 

Hp„(x, I) = i {a{x, l)x" 'Mx, 1)1", ^ ^ 



+ y {x'r{l"fBap[x,t—,-^) 

\a\=l,\p\=l 



(5.6) 
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and 



+ 2 i^"n^'x,yCap{.x',{p',,—). (5.7) 

|a|=l,|^|=l 

Here ip[t,x,rj)isasm Remark l477l 

Let f{t,x,r]) e C°°(M+ x T*{n)c), f{oo,x,r]) e C°°(r*(J7)c). We say that 
f{t,x,rf) converges exponentially fast to f{oo,x,ri] if f[t,x,ri) - f[(X),x,rf) 
satisfies the same kind of estimates as I I4.5I I . Recalling the form of we 
obtain 

v = v = 1. + (a(5:',0,,~',0)x",5|„^ 

+ 2 ^^"n^"fDap[t,x,y),—) (5.8) 

|a+y8|=2,a#0 

where Dap converges exponentially fast to some limit as f ^ +oo. We have 
on E, 

idiv(v) = ^tr A(x', 0, ri', 0) = UiF[x', 0, rj' , 0) (5.9) 
where F{x', 0, )7', 0) is the fundamental matrix of We define a{t ,x,rf)hy 

a{t,x,r])\/ dtdxdrj = a{t,x,r])\/ dtdxdrj. (5.10) 
Note that the last equation only defines a up to ii^. We have 
{Ta)^ dtdxdri — {Ta)\/ dtdxdrj 

where 

~ 5 / d 
T^ — + {A{x',{),f]',Q)x",- 



dt \ ^ ' " ' ' ' dx" 

1 ^ 

+ -xiF{x', 0, ff', 0) + pl[x', 0, ^7', 0) + Q(? , X, rj, —). (5.11) 

2 " 



Here 



^ p 

Q(f,x, rj,—)= y (x'0«(f?")^^a;5(?,x, ^7, — ) 
ox , ^x 

|a+;3|=2,a#0 
|a+^|=l 

It is easy to see that Eap and Dap converge exponentially fast to some limits 
£0^(00, X, J7) and Da^(oo,x, rf] respectively. We need the following 
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Lemma 5.2. Let A be ad x d matrix having only positive eigenvalues and 



consider the map j4 : u 



d u 

V Sxd J 



on the space P^CM^) 



of homogeneous polynomials of degree m. Then exp(f jz/)(m) = u o(exp(f A)) 
and the map j4 is a bijection except for m = 0. 

Proof. We notice that U{t) : uo exp{tA] form a group of operators and 
that (^ff^) = jz/. This shows that U{t] = exp(f jz/). To prove the second 

'^'^ f=0 

statement, suppose that u eP™, m > 1 and jz/(u) = 0. Then exp{tj^]{u) — 
u for all t, in other words uiexp{tA]{x]] = u{x], f e M, x e R'^. Since 
exp[tA){x) when t -oo, we obtain u{x) = k(0) = 0, which proves 
the lemma. □ 

Proposition 5.3. Let 

Cjix, T]) e C°°(r (fi); ^(A"-'? r(J7), A"''? r(n))), = 0, 1, . . . , 

be positively homogeneous functions of degree -j. Then, we can find solu- 
tions 

aj[t,x,r])eC'^iR+xrm,Se[A°-''r{n),A^'''T*[m), 7=0,1,..., 

of the system 115.311 with aj{0,x, rj) = Cj{x, rj), j = 0, 1, . . where aj{t,x, rj) is 
a quasi-homogeneous function of degree— j such thataj{t,x,ri) has unique 
Taylor expansions on E, for all j . Furthermore, let X{x,r]) e C(r*(J7)) and 
X\y. < miuTj, where Tj are the eigenvalues o/^trF + p^ on S. Then for all 
indices a,p,Y,j and every compact set K cE there exists a constant c > 
such that 



dJd^dPaj[t,x,r]) 



< ce 



onJ 



(5.12) 



Proof. We only need to study Taylor expansions on E. Let (x, ) be the coor- 
dinates of Proposition |4.6[ We define aj[t,x,rf) from aj[t,x,Tq) as in 1 15.101 1. 
In order to prove 1 15.121 1, it is sufficient to prove the corresponding state- 
ment for Uj (see section 1 of EO]). We introduce the Taylor expansion of do 
with respect to {x", rj"). ao{t,x, ff) = a^it,!, rf], where Sq is a homoge- 
neous polynomial of degree j in [x", rj"). Let Co(x, fj) — Xj=o ^o(^' *?)' where 
Cq is a homogeneous polynomial of degree j in [x", fj"). From Tuq = 0, we 
get a^{t,x',fj') — e"'^^5*'^^+Po)c*(x,}7). It is easy to see that for eill indices a, 
P, Y and every compact set c E there exists a constant c > such that 
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djd^dt^al < ce-f^(^'^' on M+ x K, where X{x,ri) e C{T*(n)), A|s < minzj. 

dere Tj are the eigenvalues of |tf F + pl on E. 

Again, fromfao = 0,weget(|^+j!/+|trF+Po)flo'^ (f,x,r7) = foo"'' {t,x,r]) 
where foo^^(f ,x, 17) satisfies the same kind of estimate as a^. By Lemma lS^ 
we see that exp(-rj?/) is bounded for f > 0. We deduce a similar estimate 
for the function a'f^^[t,x, rj). Continuing in this way we get all the desired 
estimates for Sq. The next transport equation takes the form TSi = b where 
b satisfies the estimates I I5.12I I. We can repeat the procedure above and 
conclude that Si satisfies the estimates I I5.12I I. From above, we see that So, 
Si have the unique Taylor expansions on E. Continuing in this way we get 
the proposition. □ 

From Proposition |5.3[ we have the following 

Proposition 5.4. Let[n-, n+), n- + n+ — n — l,be the signature of the Levi 
form. Let Cj{x, r]] e C°^[T*{ny, ^(A"-'? T*{niA°-'' T*{n))), j = 0,1,.. ., be posi- 
tively homogeneous functions of degree — j . Then, we can find solutions 

aj[t ,x,r])e C'^{R+ X rm, T*in),A^'T[m), 7=0,1,..., 

of the system i5.3i with aj{0,x,ri) = Cj[x,ri), j = 0,1,..., where aj[t,x,ri) 
is a quasi-homogeneous function of degree -j and such that for all indices 
a,p,Y,j , every 8>0 and compact setK ciJ there exists a constant c> such 
that 



dJd^dPaj[t,x,7^) 



< ce''\'^\[\ + |r7|r^'-|^l+'' onM+x ([K x M2"-i)p|5]j. 

(5.13) 

Furthermore, there exists > such that for all indices a,p,y,i and every 
compact set K an, there exists a constant c> such that 



djd^dl^aj{t,x,rj) <ce-'''\'^\[l + \ri\y^-\l^\+'^ 
onM+x(^(^:xM2"-i)p|s+j ifq^n+ 



(5.14) 



and 



djd^d^ajlt,x,rj) <ce-'''\'^kl + \ri\p-\f^\+'^ 
onM+x(^(^xM2«-i)pjj]-j ifq^n-. 

We need the following formula 



(5.15) 
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Proposition 5.5. LetQ be a differential operator on J7 of order A; > with 
full symbol q[x,^)eC'^[T*{n)). ForO<q,qi <n-l,q,qieN, let 

a{t,x,q) e C°°(l+ X T*{ny, S£{K°-'^' T{n),^°■'^ T{n))). 

Then, 



w 

\a\<k 



, (j)2[t,x,y,q) 

y=x 



where = -id^, Ra{ip,D^)a ^ {e'^^^''^'y''i^ait,y,q)} 
ix-y]ip'^it,x,q]-iijjit,x,ri)-ipit,y,q]]. 
For 0< q,qi< n - I, q, qieN,let 
ajit ,x,ri)e C^iR+ X r{ny, £e{A°-'^'T*{niA°''^ T*im), 7=0,1,..., 

be quasi-homogeneous functions of degree m — j , m G Z. Assume that 
aj[t,x,r]), j =0,1,..., are the solutions of the system I I5.3I I. From the proof 
of Proposition 15. 3[ it follows that for all indices a,p,Y,j, every s > and 
compact setKc^ there exists a constant c> such that 

djd^dllaj{t,x, q) < ce''\'^kl + |r7|r-^'-|^l+'' onM+x ((i^: x R2"-i)p|s]. 

(5.16) 

Let a{t,x,q] e C^iR+ x T*{ny, ^{A°''J^T*[n),A°'i T*{n])) be the asymptotic 
sum of aj{t,x,q). (See Definition 16. II and Remark [6^ for a precise mean- 
ing.) We formally write a(?,x, 17) ~X°lQaj(f,x,?7). Let 

{dt+df){e'1'^'-'''''^a{t,x,q)) = e''^'^'-'''''^b[t,x,q), 

where 

00 

bit,x,q]-^y^^bj{t,x,q), 

j=o 

bj is a quasi-homogeneous function of degree m+2 — j, bj e C'^(M+ x 
T*in); if (AO''?i r*(n), AO''? T*in))), 7 = 0, 1, . . .. 

From Proposition 15.51 we see that for all N, every compact set K c il, 
e >0, there exists c > such that 

\bit,x,q)\< ce"\'^\{\q\~'' + \q\'"''^~'' ilmip[t,x,qf) (5.17) 
onM+x(^(i(:xM2n-i)p|j]j_ > I 

Conversely if [dt + df )[e''l''^''''-'^^ait ,x,ri)) = e''l'^^-='-'>Mt,x,r]) and b sat- 
isfies the same kind of estimates as 1 15.171 1. then aj{t ,x,q), j = 0,1,..., solve 
the system 115. 3l l to infinite order at S. From this and the particular structure 
of the problem, we will next show 
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Proposition 5.6. Let{n-, n+), n- + n+ = n - I, be the signature of the Levi 
form. Suppose condition Y{q) fails. That is, q = n^ orn+. Let 

aj[t,x,r])eC°^iR+xr[n);Se[A°-Tin),A'''''T*[m), 7=0,1,..., 

be the solutions of the system (5.3^ with ao[0,x,r]) = I, aj[0,x,r]) = when 
j >0, whereaj{t,x,ri] is a quasi-homogeneous function of degree -j . Then 
we can find 

aj (00, X, 77) e C°°( r (fi); if (A"'"? r (f2). A"''? r (n))), ; = 0, 1 , . . . , 

where aj(oo,x, rj) is a positively homogeneous function of degree -j, So> 
such that for all indices a, l3,Y,j, every compact set K cil., there exists c> 0, 
such that 

dJd^dP{aj{t,x,r))-aj[oo,x,q)) <ce-'''\A^ + \q\r-\f'\+'^ (5.18) 

onl+x(^(ii:xE2«-i)p|j]j^ > 1. 
Furthermore, for all j — 0,1,..., 

{all derivatives of aj[oo,x, q) vanish at E+, ii q = n_, n- 7^ n+ 
all derivatives of aj[oo,x, q) vanish at S~, if q — n+, n- / n+. 

(5.19) 

Proof We assume that q = n^. Put 

a{t,x,q)-^y^^ajit,x,q). 
J 

Since aj{t,x, q], j = 0,1,.. ., solve the system 1 15.31 1, we have 

(5f +nSf^Xe''^'^''^'^'a(f,x,r?)) = e''^'(^'^'^^fo(f,x,r?), 
where b[t,x,q) satisfies 1 15.171 1. Note that we have the interwing properties 

Ob — Qfo Cb 
Now, 

I ^b{e''''a] = e''Pd, 
a^Y!jL-iSiit,x,q), d^Y!jL-i^jit,x,ri), where 

dj e C°°(l+ X T*{n]; ^(A°''? r(J7), A''''?+i r*(J7))), 7 = 0, 1, ... , 



37 



Uj e c°°(E+ X nny, ^(A"''? r(j7), A"-'?-! T*[n]]], j = o,i,..., 

and dj, Sj are quasi-homogeneous of degree I- j. From II5.20II . we have 

where b\, bz satisfy I I5.17I I. Since b^, bz satisfy I I5.17I I, Sj , dj , j = 0,1,..., solve 
the system II5.3I I to infinite order at E. We notice that q —l^ U-, q + \^ U-. 
In view of the proof of Proposition |5.3i we can find Eq > 0, such that for all 
indices a, 13,^, j , every compact set K cil, there exists c> such that 



djd^d^ajit,x,iq) 
dJ d^dll dj{t,x,r]) 



< ce-^»'hl(l + |}7|)i-J'-|^l+'' 

< ce"''°'l''l(l + |j7h^"^'"l'^l+'' 



(5.21) 



onM+x(^(ii:xM2n-i)p|j]-j^ > I 



NowD^^^-" = dbdb +db db,so n^^^Xe'^^a) = e'^c, where c satisfies the same 
kind of estimates as II5.21I I. From this we see that dtie'^^a) = e^^'d, where d 
has the same properties as c. Since d = i{dtip]a + dtU and dtip satisfy the 
same kind of estimates as II5.21I I, dta satisfies the same kind of estimates as 
I I5.21I I. From this we conclude that we can find a[oo,x, rj] ~ X°lo (ijioo,x, rf), 
where aj{oo,x,rj) is a matrix-valued C°° positively homogeneous function 
of degree > 0, such that for all indices a, p,Y, j and every compact 

set c Jl, there exists c> such that 



djd^d^iuj {t,x,r]]- ajioo,x,r]]] 



< ce 



\+r 



on 



x(^(iS:xM2n-i-jp5]-j^ > I 

If n- = n+, then q — l^ n+, q + \^n+. We can repeat the method above 
to conclude that we can find a[oo,x, J?) ~Xjlo '2j(oo,x, 17), where aj(oo,x, 17) 
is a matrix- valued positively homogeneous function of degree — j, Sq > 
0, such that for all indices a, p, y, j and every compact set Ken, there 
exists such that 



on 



(^(^xM2n-i-jp5]+j^ > I 

Now, we assume that n- / n+. From II5.14I I. we can find Sq > 0, such that 
for all indices a, p,Y, j and every compact set K cil, there exists c> such 
that 

dld^"d^Pajit,x,r]) < ce-'"'\''\il + \r]\rHP\+r 
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onM+x(^(ii:xM2n-i)p|j]+j^ > I The proposition follows. □ 

6 Some symbol classes 

We continue to work with some local coordinates x = (xi,... ,X2„-i) defined 
on an open set ncX.We identify T*in) with n x R^"-\ 

Definition 6.1. Let rix,ri) be a non-negative real continuous function on 
r*(n). We assume that r{x,ri] is positively homogeneous of degree 1, that 
is, r{x,A.r]) — ?irix,ri], for A > 1, |}7| > 1. For 0<qi,q2<n-l, qi,q2 e N and 
A; e M, we say that 

if a e C°°(M+ X T*{p); ^{K^-i^ T*[n),A°-i-' T*{n)]) and for all indices a, I3,r, 
every compact set ^ c f2 and every e>0, there exists a constant c> such 
that 

dld^^dPa[t,x,q) < ce'^-'^''''^^+'\'^\\l + \ri\)''+r-\P\^ x e K, \r]\>l. 
Remark 6.2. It is easy to see that we have the following properties: 

(a) If a e Sf^ , e §1^ then ab e a + be S^J'^)- 

(b) If a eSf then ^/^/^^^a esf"'^'^''. 

(c) If Uj e S/, 7 = 0,1,2,... and kj \ -oo as j oo, then there exists 
a e such that a - Uj e S^" , for all f = 1, 2, . . .. Moreover, if S^°° 
denotes HiteK'^r ^^^^ is unique modulo S~^. 

If a and Uj have the properties of (c), we write 

oo 

a ~ ^ a j in the symbol space S^" . 



From Proposition 15.41 and the standard Borel construction, we get the 
following 

Proposition 6.3. Let 

Cjix, q) e c°°(r(n); ^(A"'" r(n), A"'-? T%n))), 7 = 0, 1, . . . 

be positively homogeneous functions of degree - j. We can find solutions 
aj[t,x, q) e C°^{R+xT*[n); ^[A°-i r*(J7), A"-'? T*{n))), ; = 0, 1, . . . of the system 
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f5.3D with the conditions a j{0,x, rj) = Cj[x, rj), j = 0,1,.. ., whereaj isaquasi- 
homogeneous function of degree -j such that 

ayeS7(E+x r(n);if(A°''?r(n),A°''?rTO), ;=0,1,..., 

for some r with r>0 if Y[q] holds and r = if Y{q) fails. 

If the Levi form has signature{n-, n+), n- + n+ = n — \, then we can take 
r >0, 

near E+, iiq = n-,n-^n+, 
near Tr, if q = n+,n-^ n+. 

Again, from Proposition |5.6l and the standard Borel construction, we get 
the following 

Proposition 6.4. Let{n-, n+), n- + n+ = n- I, be the signature of the Levi 
form. Suppose condition Y[q) fails. That is, q = n- or n+. We can find 
solutions 

a^(f,x,r7)ec"(l+x r(n);if(A°''?r(n),A°''?r(n))), ; = o,i,... 

of the system II5.3I I with aQ[Q,x,rj) = /, aj[Q,x,rj) = when j > 0, where 
aj[t,x,r]) is a quasi-homogeneous function of degree— j, such thatforsome 
r>0 as in Definition \6.1[ 

aj{t,x, q) - a; (00, x, q) e S-j^+ x T*{n); ^(A"''? T* (J^), A°''? T* (fi))), 

=0, 1,..., where 

ay (00, X, T]) e C°°(r (J7); ^(A"''? T*m, A"-'? r{n])], ; = 0, 1, . . . , 

and ajioo, x, q) is a positively homogeneous function of degree -j . 
Furthermore, for all j = 0,1,..., 

aj{oo,x,'q) = in a conic neighborhood of , iiq = n-,n-^n+, 
aj[oo,x,ri) = in a conic neighborhood of T.~ , iiq = n+,n^^n+. 

Let b{t,x, q] e S^, r > 0. Our next goal is to define the operator 

as an oscillatory integral (see the proof of Proposition |6.5[ for the precise 
meaning of the integral B{t,x,y)). We have the following 
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Proposition 6.5. Let 

with r > 0. Then we can define 

B{t] : C^{n; A"''?! T*{n)) C~(M+; C°°(f2; A°* r(f^))) 

it;zr^ distribution kernel B{t,x,y] = ^ e''''^''''''''^^^^^''^)^b{t,x,r])dr] and B{t) 
has a unique continuous extension 

Bit) : SXn; A"-'?! T*in)) C~(M+; ©'(fi; A"'*?^ r*(n))). 

We have 

B{t,x,y) e C~(R+; C~(n x \ diag(fl x Jl); ^(A"-'?' r(fl), A"'* r(n)))) 
awrf B(f,x,y)|(>o e C°°(M+ x x fl; ^(AO>'?i r*(n), AO''?^ T*{n))). 
Proof. Let 

S*J7 = {(x,r7)ef^xE2«-i; |r7| = l}. 

Set S*S = E fl S*n. Let ^ c M+ x S*J^ be a neighborhood of (M+ x S*E) |J(|0} x 
S*n) such that K = {[x, rf); {t,x,rf)&V} is independent of t for large t. Set 
W = |(f,x,r;)eM+ xnx]R2"-i; z-,x,i^)e y|. Let Xv e C°°(M+ x 8*9) 
have its support in V, be equal to 1 in a neighborhood of (E+ x S*S) |J(|0} x 
S*J7), and be independent of for large t. Sei Xw{t ,x ,rii) = Xv(\'n\ ^'-^'i^)^ 
C°°(E+ X X M2"-i). We have /M,(f,x,A}7) = Xw{^t,x,r]], A > 0. We can 
choose V sufficiently small so that 

\iP'^{t,x,r])-r]\<^-^mW. (6.1) 

We formally set 



r 



Bit,x,y] = 



r 

+ 



mt.^.nHy.n)\l_^^^t^^^r])Mt,x,r])dr] 
^imt.x.n)-{y.r,)) ^^^t,x, r]Mt,x, r])dr] 



= Bi{t,x,y) + B2{t,x,y) 

where in Bi{t,x,y] and B2{t,x,y) we have introduced the cut-off functions 
(1 - Xw) and Xw respectively. Choose x £ C^(]R2«-i) so that xiv) = 1 when 
Ir^l < 1 and xi^) = when \r]\ > 2. Since hmp > outside (R+ x E)|J({0} x 
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M^" we have lmip{t,x,ri) > c^ri^ outside W, where c > 0. The kernel 
-Si,f(?,x,j/) = ]'e'W'(f'^''')-(>'''?))(l-j^v(r,x,J7))Z7(f,x,)7)/(£^)d)7 converges in 
the space C^iW+xnxn; ^[hP-'^^ r*(J^), A"-* T*{n))) as f ^ 0. This means that 

Bi{t,x,y) = \imBi,,{t,x,y) e C°°(M+ xnxfl; S£{!^''^' r(n),A°* r(J7))). 

f— >o 

To study B2[t,x,y) take a a(y) e C^iK; A"-'?! r*(J^)), ^ cc and set Xv{n) = 
/(2-'')7)- / (21-^)7), V > 0, x^W> = Then we have Xr=o/' = ^ ^'^d 

2^ -1 < |}7| < 2^+1 when r] e supp/,,, v 7^ 0. We assume that h{t,x,r]) = if 
|}7| < 1. If X e i<r, we obtain for all indices a, p and every s >0, there exists 
Ce.a.p.K > 0, such that 

D"^DPiXriri]Xwit,x,riMt,x,r]))\<c,,a.p.Ke'^-'^'^^^^^^^ (6.2) 

Note that |-D";tfv()7)| < Ca(l + |}7|)"l"l with a constant independent of v. We 
have 



£'^(;rvv(f,x,2'}7)fo(^x,2^)7)) 



< 



where A = 2^'. Since I I6.2I I holds, we have 

c2'''' if X e iC, 1< |r7| < 4, where c> 0. Since dyiipiXt ,x,r]]- {y ,r])] ^ 0, if 
J7 7^ 0, we can integrate by parts and obtain 



|B2,v+i| < c2^'(2"-i+'^-'«) ^ sup|D"M|. 



\a\<m 

Since m can be chosen arbitrary large, we conclude that | B2,^ | : con- 
verges and that B{t] defines an operator 

B{t] : C^iny] A°'^i rim C°°(M+ X n^; A"'* r(J^)). 

Let be the formal adjoint of B[t] with respect to ( | ). From I I6.1I I. we see 
that xp'J^t,x, 57) 7^ on W. We can repeat the procedure above and conclude 
that B*[t) defines an operator 

B*{t) : C^[n^; A"'* T*in]] C°°(M+ x Uy-, A°''?' T*in]]. 

Hence, we can extend B{t) to A°'i^ T*{n]) C'^{R+; S>'{n; A"-* T*{n))) 
by the formula 

mtMy)\vix))^[u{y]\B*{tMx)l 
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where u e S'in; A^-i^ T*{n)), v e C^in; A"'* T*in)). 

When x^y and (x,y ) e S x E, we have dr^{xp{t,x, r])- (y , J?)) / 0, we can 
repeat the procedure above and conclude that B{t,x,y) e C°°(M+; C°°[il. x 
n \ diag(J^ X n); if (A''>'?i r*(n), A"'* T*in)))). 

Finally, in view of the exponential decrease as f ^ oo of the symbol 
b{t,x, rj), we see that the kernel B{t)\t>o is smoothing. □ 

Let b{t,x, rj) e with r > 0. Our next step is to show that we can also 
define the operator -B(x,y ) = J e'^-^'-'^-^'^^'^^-^'^^bityX, r])d t j drj as an os- 
cillatory integral (see the proof of Proposition |6.6[ for the precise meaning 
of the integral B{x,y)). We have the following 

Proposition 6.6. Let 

b{t,x,r])eS'^X^+ x r(J^); ^(A"-'?! r(J^),A°*rM 
with r > 0. Assume that bit, x,ri) = when |j7| < 1. We can define 

B : C^in; A"-'?! T ^ C^{n; A°'* T 
with distribution kernel 

and B has a unique continuous extension 

B : SXn; A"''?! T*{n]] A°'* r(JI)). 

Moreover, B[x,y] e C^in xn\ diag(J7 x n); if (A"-'?! r*(J^), A"-* 
Proof. Let and/i^(f,x,r7) be as in Proposition l6.5[ We formally set 

B(x,y) = ^^^^JJ^ e'^'^'^'''''^^-iy'^)\l-Xwit,x,r^)Mt,x,r])dtdr] 

= Bi(x,}/)+B2(x,y) 

where in Bi{x,y) and B2(x,y ) we have introduced the cut-off functions (1 — 
Xw) and Xw respectively. Since lmxp{t,x,r]] > c'^rj^ outside W, where c' > 
0, we have 

e'Wt.x.r,)-{y.ri))f^l_^^f^t,x,r])Mt,x,r]) < ce-''\''\e-"''\''\{l + \r]\f , So>0 
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and similar estimates for the derivatives. From this, we see that Bi{x,y)' 



Choose / e C° 



oorTn>2jj-n 



so that ;f (ry) = 1 when m < 1 and /(ry) = when 



> 2. To study Bzix.y) take a u(y ) G C^CiC; AO"?! r*(J^)), cc J2 and set 



1 



(271)2"-! 

We have 
B 



( 



-J 



,iiipit,x,r])-{y,ri 



)^b{t,x,ri)xw{t,x,r])x{j)u{y)dydr]^dt. 



{xi2^-Xil))u{y)dydTq jdt. 
Since dy[ip[?^t,x,ri) — (y,)7))/0, 17 / 0, we obtain 



< cA-~ 2 sup 

|a|<Af 

<c'A-^e-^°H^I(l + |A|)^ 



^",^Jw(?,x,A)7)f7(f,x,A}7X/(-)-/()7))M(y) 



where c, c', > 0- Hence 52(x) = limA^oo B2,a(^) exists. Thus, B{x,y) de- 
fines an operator B : C^CJ^^,; A"-'?! r*(n)) ^ C°°(n;,; A"-* r*(l^)). Let B* be 
the formal adjoint of B with respect to ( | ). Since xp'J^t,x,r]) 7^ on W, 
we can repeat the procedure above and conclude that B* defines an op- 
erator B* : C^{nx;hP'''^T*{n)) -» C~(nj,;A0>'?i7^(n)). We can extend B to 
A"-'?! r*(n)) -> QJ'[9.; A''''?^ r*(fl)) by the formula 

{Bu{y)\v[x)) = {u{y)\B*v{x)), 

where u e S'{n; hP-'^^ T*{n)\ v e C^i^; A"'* r*(J^)). 

Finally, when x^y and (x,y ) e E x S, we have dn[xp{t,x, ri)-{y ,ri))^0, 
we can repeat the procedure above and conclude that B{x,y) e C°°(ri x Jl \ 
diag(J^ X J^); ^(A"-'?! r*(r?), A"--?^ □ 

Remark 6.7. Let fl(r,x, r^) e Sj(M+ x r*(f2); .^(AO'"?! r*(J2), A^'* T*{n)]]. We as- 
sume a(?,x,}7) = if jr^j < 1 and 

a{t,x, rj) - fl(oo,x, r}) e S^(R+ x r(n); .S?(A''''?' r(fi), A''''?^ 
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with r > 0, where a(oo,x, r]) e C°°(r*(J7); 5eiA°-'J^ r*(J7), A"-* r*(J^))). Then we 
can also define 

as an oscillatory integral by the following formula: 

We notice that {-tXiip[it,x,ri)ait,x,ri) + a[it,x,r]))eS''^+\ r >0. 

Let B be as in the proposition 16.61 We can show that B is a matrix of 



pseudodifferential operators of order A; - 1 type We review some facts 



2' 2'' 



about pseudodifferential operators of type (-, -). 

Definition 6.8. Let fc e M and q eN. i(r*(JI); ^iA^-^i T*in),A°-i T*in))) 
is the space of all a e C°°{T*[n); Se[A°-i r*(jl), AO"? T*{n))) such that for every 
compact set K en and all a e e N^^-i, there is a constant Ca,p,K > 

such that d;^dfaix,Q < c„,^,jf(l + (x,<^)e T*{n), xeK.S't, is 

^ 2'2 

called the space of symbols of order k type [^A]. We write 57°^ = HmeK ^Ti' 

2'2 2'2 

OOO I I cm 



2'2 2'2 

2'2 



Let a(x, f ) e S*^ i (r*(n); ^(A"-'? r*(J7), A"-'? T*in])]. We can also define 



as an oscillatory integral and we can show that 

A : C^{n; A"'*? r (O)) ^ ^^(5^; A°'^ T* (J^)) 
is continuous and has unique continuous extension: A : A^'i T*(SY}) 

Definition 6.9. Let k gR and letO < q < n — I, q gN. A pseudodifferen- 
tial operator of order A; type (|, |) from sections of A" '? r*(Jl) to sections of 
^o,q j*^^^ is a continuous linear map A : C^[n; A^-'^ T*{n)) A"'^ T*{n)) 

such that the distribution kernel of A is 



Ka 
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with a e St 1 (r*(n); if (AO'-? T*[n],A°-i T*[n))). We call a[x, the symbol of 

2'2 

A. We shall write i A°-i T*[n),A°-i T*[n)) to denote the space of pseu- 

2'2 

do differential operators of order k type (|, |) from sections of A"'? r*(f2) to 
sections of A".'? T*m We write LTT = H^eM ^™ i - i = UmeM ^ ■ 

2'2 2'2 2'2 2'2 

We recall the following classical proposition of Ccilderon-Vaillancourt 
(see chapter of Hormander HH]) 

Proposition 6.10. If A e L\ A"-'? r*(n), A">'? T*{n)). Then, 

2'2 

A : H'„^p(n; A"-'? T* (J^)) ^ Hf-*^(n; A"-'? T* (J^)) 

/5 continuous, for all 5 e M. Moreover, if A is properly supported (for the 
precise meaning of properly supported operators, see page 28 oflWH), then 

A : H'^jn; A"''? Tm ^ H'^-^n; A°''? T* 

J5 continuous, for all 5 e M. 

We need the following properties of the phase ip{t,x, rj). 

Lemma 6.11. For every compact set K c and all a e N^"~\ p e N^"-i, 
|a| + 1^ I > 1, there exists a constant Ca,p,K > 0, such that 

a , > II \a\+\l3\ 

dy^Pmt,x,r])-{x,r])) <c,,p,Ka + \r]\)—[l + lmiPit,x,r]))—, 
if\a\ + 1^1 = 1 and 

dyPmt,x, r?)- (x,r7))| < c„,^,j,(l + if \a\ + \p\>2, 

where X gK, t G E+, [ryj > 1. 

Proof. For |}7| = 1, we consider Taylor expansions of dxj{ip{t,x,r]]- {x,ri)), 
= l,...,2n- 1, at(xo,)7o)eS, 



dx {xp{t,x,r])- {x,r])) = \- — - — {t,Xo,r]o){Xk- x^^^) 
' ^dxkdxj 

j-j-{t,x^,ri^\rik-rtQ 

+ Oi\ix-xof + \iri-rjo)\\ 

where Xq = (Xo^\ . . .,x'^"~^^), r]o = (}7g^\ . . . , 170^""^^). Thus, for every compact 
set i<C c there exists a constant c> 0, such that 

\dx{ip{t,x,ri)- {x,r])]\ < c--^dist((x, 17),^)), 

J- l~ I' 
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where xe K, t eR+ and |}7| = 1. In view of II4.4I I, we see that lmip[t,x,ri] 
(^)dist((x,r7),S))2, \r]\ = 1. Hence, (^)5dist((x,r7),S)) (Im?/'(f,x,)7))5, 
|}7| = 1. Thus, for every compact set K c n there exists a constant c > 0, 
such that \dx{ip{t,x,r])- {x,r]))\ < c[^f4lmip{t ,x,r]]]K \r]\ = 1, x e K. 
From above, we get for m > 1, 



\dAipit,x,r])- {x,r]))\ = \r]\ 



< c m r ( 



t 


r] 




l + t\ 


r]\ 



'-ilmip{t,x,ri)) 



< c'{l + \r]\)2{l + lmipit,x,r]))2, 

where c, c' > 0, x G K, t G IR+. Here K is as above. Similarly, for every 
compact set c there exists a constant c> 0, such that 

\dr,{xpit,x,r])- {x,ri))\ < c(l + |?7|r^(Ini?/)(r,x,)7))5, 

where xG K, t eR+ and m > 1. 



Note that 



d;^d^^mt,x,ri)-{x,ri)) 



For |a| + > 2, we have 



is quasi-homogeneous of degree 1 



d;^d,^mt,x,r])-{x,r])] < cil + \ri\Y~\P\, 



where c > 0, x e K, t gR+ and \ri\ > 1. Here K is as above. The lemma 
follows. □ 

We also need the following 

Lemma 6.12. For every compact set K c Q. and all a e N^""!, e N^""^ 
there exist a constant Ca,p,K > and e>Q, such that 

dyP{til:[{t,x,r))) <Ca.pA^ + \r]\)—e-''\^\{l + lmil^{t,x,r))f+— 

if\a\ + \j5 \ < 1 and 

d^dP{txP\{t,x,r^))\<Ca,pA'^ + \ri\f-\l'\e-''\^\ 

if\a\ + \fi\ >2, wherexeK, teR+, \ri\ > 1. 

Proof. The proof is essentially the same as the proof of Lemma lGJTl □ 

Lemma 6.13. For every compact set K c and all a e N^"-!, p G N^"-!, 
there exist a constant Ca,p,K > and s > 0, such that 

(6.3) 
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and 



"1+1/31 



(6.4) 



d^"dJ^[e'^^^'-''-'^^-('''''hip[it ,x,ri]] 
where X gK, t G R+, |}7| > 1. 

Proof. First, we prove J6.3I I. We proceed by induction over |a| + |^|. For 
|a| + < 1, from Lemma leTTl we get Let \a\ + \p\>2. Then 



a'+a"=a,p'+P"=li,(a" ,P")^Q 



d^"'dfie'^^^''''''^^-(''-''))dfdfiiipit,x,ri]-i {x,ri)) 



X q 



c > 0. By the induction assumption, we have for every compact set K cil, 
there exists a constant c> 0, such that 



ga'gp'^^iiipit,x.n)-{x.n)^ <c(l + |nh^^e-^"^'^f''^'''^(l + lm?/)(r,jc,n)) 

XT] II 



(6.5) 



where xeiC, feE+,|)7|>l. From Lemma [6.11[ we have 

\c"\-\l}"\ . ^ 



df'df'{iijjit,x,r])-i{x,r])] < c(l + 1)7|)^^(1 + Im?/^(r,x,}7))- 



where xG K, t e E+, |)7| > 1. Combining 1 16.51 1 with I I6.6I I, we get 
From Leibniz's formula, Lemma [6.12l and 1 16.31 1, we get II6.4I I. 



(6.6) 



□ 



Lemma 6.14. Letb{t,x,ri) e Sf(M+ x T*iny, ^[A^'^i T*{n),A°-i T*{n))) with 
r >0. We assume that b{t,x,rj) = when^rj^ < 1. Then 

e'Wt.x.r,)-{x.n))m^x^^^af 





2'2 

Proof. From Leibniz's formula, we have 



< 



^ S {d^"'dfe'^^^'''''^^-('-'^))[dfd;/"bit,x,r]]] 



a'+a"=a,p'+p"=p 



, OO. 
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From I I6.3I I and the definition of S^, we have for every compact set K ciJ, 
there exist a constant c> and s >0, such that 

f 00 

<c e-''^'^^'-''-'^\l + \r]\f+'^[l + lmip[t,x,r])]^e-''\'>\dt 



<c'(l + |r7|) 

where c' > 0, x G K. The lemma follows. □ 

We will next show 
Proposition 6.15. Let 

b{t,x, ri) e (1+ x r{9); ^(A°'^ r (J^), A°''? r{9))) 

with r > 0. We assume thath[t,x, rf) = Q when | J7| < 1. LetB he as in Propo- 
sition\6M Then B e L\~lin; A"-'? r*(J7), A*'-'? r*(J7)) ^ymZ^o/ 

2'2 

J> OO 
e'(^'('-^-^H^-'^)^b{t,x,'n]dt 




s'l'lirin); if (A"''? T*{_n),A°''i T*in))). 

2'2 



Proof. Choose x e C^(M^""^) so that /()7) = 1 when |)7| < 1 and j()7) = 
when > 2. Take a M(y ) e C^in; A"'-? r*(J^)), then 

5u = lim j^^^, ( J , X, r])u[y)xier])dri') d t 

= lim^^^^| e'(™)(J e^^'f^'''-^H--^)Mt,x,r^)uiy)xisr]))dr^dt 

From Lemma [6.14[ we know that <7(x, 17) e S^~/- Thus 

2'2 

Urn— 4^ I e'(^-^''')^/(x,r;)u(y);f(fr7)dr?eL^Hf^; A"'''r(n),A°''?nf])). 

□ 

We need the following 
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Lemma 6.16. Let a[(x>,x,r]) e C'*(r*(J7); if(A0''?r*(fi),A''''?r*(J7))) beaclas- 
sical symbol of order k, that is 



a{oo, X, rj) ^ ajjoo, x, 17) 



7=0 



in the Hormander symbol space SlQ{T*{n); ^[A°'iT*{n),A'^-'iT*in))), where 

ay (00, X, T]] e C^iTin); if (A"''? r(J7), A"-'? r[n))), j=0,l,..., 

aj[oo,x,?^ri) = aj{oo,x,r]), A > 1, |)7| > 1, 7 = 0,1,.... Assume that 
a{oo,x,r]) = when\ri\ < 1. Then 

J> 00 
^^mt.x.r,)-{x.n)) _ giWcc,x,r,)-{x.r^))^ ^^^^^ ^ 


e Si"/(r*(j^); if (A"''? r*(n), AO'*? r*(j^))). 

2'2 

Proo/ Note that 

f 00 

From 1 16.411 . we can repeat the procedure in the proof of Lemma l6.14l to get 
the lemma. □ 

Remark 6.17. Let a{t,x, r]) e S^(M+ x T*in); if (A0>'? T*[n),A°-i T*{n))). We as- 
sume a{t,x,r]] = 0, if |}7| < 1 and 

a[t,x,r]]-aioo,x,r]]eS''^iR+ x T*m, ^{A°'iT*m,A°-'^T*m]] 

with r > 0, where a((X),x,}7) is as in Lemma [6.161 By Lemma [6.141 and 
Lemma [6.16[ we have 

J> 00 


J> 00 fOO 
e'Wt.x.r,)-{x.r,))(^^^f^X,r]]-a{00,X,r])]dt+ ^eiWt.x.rj)-{x.rj)) 
Jo 

y 2'2 

Let 

1 



J> 00 
^e'^4>it.^^My-n)^a{t,x,r]) 




,iiip[oo,x,r])-{y 



'^'>^aioo,x,if]]jdtdri 
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be as in the Remark |6.7[ Then as in Proposition 16.151 we can show that 

A e L'[-^in; A"-'? r*(J^), AO''? T*in)) with symbol 

2>2 

J> 00 




: s'i~\T*{n); se[A°-i T*{n),A°-'' T*{n]]). 



1 i 

2'2 

We have the following 



Proposition 6.18. Let a{oo,x,rj) e C^iT*iny, Se{A'^'iT*{n),A°-''T*{n))) be a 
classical symbol of order k . Then 

a{x, r])^e ' W'('^'^'''^-(^''')'a((X), x, r]]eS'{,{ T*{n); if (A"-'? r{n), A"''? T*{n)]). 

2'2 

Proof. In view of Lemma [6.13[ we have for every compact set K cD. and all 

a e N^""!, p e W""^, there exists a constant Ca,i5,K > 0, such that 

gyPf^^iWco.x.r,)-{x.r,)^ < C„,^,jf (1 + | )^ e-^'"'^'°°'^'''\l+Im ?/>((X), X, r/))^ , 

where x e iC, |}7| > 1. From this and Leibniz's formula, we get the proposi- 
tion. □ 

7 The heat equation 

Until further notice, we work with some local coordinates x = (xi, . . . , X2„-i) 
defined on an open set ^IcX. Let 

b{t,x, r]] e (E+ x T*[n]; if (A°''? T\niA°'i r[n])] 

with r > 0. We assume that b{t,x,ri) = when ^r]^ < 1. From now on, 

we write j^^^^/(^/J°e'^'^^'^'''''^'"^^''^)^fo(r,x,r7)d?jd^ to denote the kernel 

of pseudodifferential operator of order A; - 1 type (|,^) from sections of 
AO'-? T*{n] to sections of A"-'? T*in) with symbol 

J' 00 
^im.x.r,)-{^.r,))i,(^t,x, rj)d t e S'',-\T*{n); if (A"'-? r(fi). A"-'? T*[n))). 

(See Proposition |6.15[ ) 

Let a{t,x, r])eS^iR+ x T*[n); if (A"''? T*[n),A°-'i T*[n])]. Assume that 

a{t,x,ri) = 

when |}7| < 1 and that 

a(f,x,r;)- a((X),x,r;) GSf(l+ x 7%^); J^iA^-'^ T*{rt),A°-'' T*{n))) 
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with r > 0, where a{oo,x, -q] e C'^(r*(J]); if (A"-'? T*{n),A°''' T*{n))) is a classi- 
cal symbol of order k. From now on, we write 



(271 



to denote the kernel of pseudodifferential operator of order A; - 1 type (-, -) 
from sections of A"'*? T*{iY} to sections of A" '? T*{iY} with symbol 

J> 00 


in S'rX{T*[n); S£{JSP''i r*(J7), A"-'? T*{^1))). (See Remark [6J7l ) From Proposi- 

2'2 

tion |5.5[ we have the following 

Proposition 7.1. LetQ be a dijferential operator on of order m . Let 

b{t,x, ri) e (1+ x T*in]; if (A°'^ r (^J), A°''? T*[n]]] 
with r > 0. We assume thatb{t ,x,r]) = when |?7| < 1. Set 

Q(e'W'(f'^'^^-(>''^)'fo(Z:,x,r7)) = e'W''''^''^^-<^'^))c(Z:,x,r7), 
c(f,x,r7)eS^+™(M+ x T*iny, SeiA'^-i T*in),A°-'i T*[n))), r > 0. Put 

Bi^'y^ ^ (^^^ J ( e'^^'^'-^'-^^-iy'^Hit.x, r])d t) dr^, 

Proposition 7.2. Lef Q i>e a C°° dijferential operator on Jl of order m . Let 

b{t,x,ri)&Sl^+ X r(J7); ^(A°''?r(J^),A°''?r(J7))). 

We assume that b[t,x,r]) = Q when\rj\ < 1 andthat 

b{t,x, T]] - b{oQ,x, T]] G Sf (R+ X r(n); if (A"''? r(n), A°'^ r(r2))) 

with r>0, where b{oo,x,ri) e C°°(r*(n); i£{K^-'^T*[n),AP-'iT*[p))) is a classi- 
cal symbol of order k. Set 

Q(^e'^'^f^'^''^^-<^''')'fo(^x,r?)-e'W'(~'^''''-<^''')^fo(oo,x,r7)] 
= e'W'(f'^.'7)-(y.'?))c(f,x,r7)-e'W'('^''''''^-^^''')^c((X),x,^), 
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where cit,x,ri)eS'^+"'iR+ x T*in); SeiA^'^i T*in),A°''f T*in))), 

c{oo, X, r]) e C°°iT*{n); if (A°''? T* A"''? r (fi))) 
is a classical symbol of order k + m. Then 

c{t,x, ri) - c{oo,x, ri) e Sf+'"(M+ x r ^(A"-'? r(n), A"''? r[n))) 
with r > 0. Put 

1 



(271) 



2n~l 

J 



J> oo 
(^e'^^'i'-^-nHy-n))b{t,x,r])- 


C(x,y)=^^^ J(J^ [e'^^'^^'-^Hy^^)^cit,x,rj)- 

Wcc.x.r,]-{y.r,))ci^^^x,r]ijdt^dr]. 
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WehaveQoB = C. 

We return to our problem. From now on, we assume that our operators 
are properly supported. We assume that Y[q) holds. Let 

ajit,x,r])e S;J{R+ x r(n); ^(A"-'? r(n), A°''? r(n))), ; = 0, 1, . . . , r > 0, 

be as in Proposition |6.3| with ao{0,x,ri) — I, aj{0,x,ri) = when ; > 0. Let 
ait,x, T]] ~ XJIq ajit,x, ri) in S%R+ x T*{n); if (A"'"? T*iniA°'i T*{rt)]), where 
r > 0. Let 

(^f+n!f^)[e''('^^''^'^^-<>'''')^a(f,x,r;)]=e'W'(f'^''')-(j'''?))Z7(Z:,x,r7). (7.1) 

From Proposition l5.5[ we see that for every compact set c Jl, f > and all 
indices a, p and iV e N, there exists Ca,p,N,s,K > such that 



dyPbit,x,r]] 



(7.2) 

where f G E+, x e iC, |}7| > 1. Choose j e C^(]R2«-i) so that xi^)^'^ when 
jr^l < 1 and /(}7) = when |)7| >2. Set 

A(x,y)= ^^^|^^_^ J(J^ e'^^'^'-'''^Hy-^)^ait,x,r]]il-Xmdtyr]. (7.3) 
We have the following proposition 

Proposition 7.3. Suppose Y{q) holds. Let A be as in i7.3d . We have \li^^A = I. 
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Proof. We have 

where b[t,x,ri) is as in II7.1II . I I7.2I I. From Proposition l7.1[ we have 
L_ J J ^)(1 -xmdt]dr^ 

From lO, it follows that f (^f^ e'^^^''''-'^^-(y-'')^bit,x,ri)il - xiriMt^drj is 
smoothing. Choose a cut-off function /i(}7) e CJ^(M^"~^) so that /i(j7) = 1 
when \ri\ < 1 and Xiil) = when |r7| > 2. Take a M(y) e C;^(J7; A0''?r*(J7)), 
then 

= -lim JJe'(--^'^)(l - ;r(^))/l(^^)"(y)^^^^^y• 
Hence^^/(J;"|^(e'W'(^■-■'')-(3'.'?)^fl(^,x,^^^ Thus 

Remark 7.4. We assume that holds. From Proposition 17.31 we know 
that, for every local coordinate patch Xj, there exists a properly supported 
operator : ^'{Xj; A''-'? T*iXj)) ^'{Xj; A''-'? T*iXj]) such that 

Aj : H^JXj; A''^ T*iXj]) ^ Hf+H^;-; A^'" nXj)) 

and oA^. - / : H^^^iXj; A^-''T*{Xj)) Hf+^CX^; A<'''?r*(Xy)) for all 5 e M 
and m > 0. We assume that X = {j'^j^-^Xj. Let |/^| be a partition of 
unity subordinate to [Xj] and set Aw = Y^jAjiXjU), u e 9'[X; AP-'^T*[X)). 
A is well-defined as a continuous operator: 



A : H\X; A"''? T* (X)) ^ H^+^X; A°''? T* (X)) 

for all 5 e M. We notice that A is properly supported. We have D^^^ oA — 7 : 
W{X; AO'^ r*(X)) ^ H^+'"(X; A"''? r*(X)) for all 5 e M and m > 0. 
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Assume that Y{q) fails. Let 

ajit,x,r])eS~^\R+ x T*m, S^{A°-i T*{niA°-T[mi 

= 0, 1, . . ., and Ujioo, x, r]) e C°^{T*[ny, ^(A"''? r*(0), A"-'? T*{n))), j = 0,l,.. ., 
be as in Proposition |6.4[ We recall that for some r>0, 

aj{t,x, r]) - aji(X),x, r]) e S;^(M+ x T%n); ^(A"-'? r(n), A°'^ T*{n))), 

=0,1,.... Let 

a(oo,x,)7) ~^aj(oo,x,)7) (7.4) 

7=0 

in the Hormander symbol space S%{T*{ny, ^(A"-'? T*{n],A°-'J T*{n)]). Let 

00 

a[t,x,ri)-^'^ajit,x,ri) (7.5) 

i=o 

in S°o{R+ X T*{n); i£{A^''^ T*[n),A°-i T*[n))). We take a[t,x, r]) so that for ev- 
ery compact set K (Z and all indices a, p, j, k, there exists c > 0, c is 
independent of t, such that 

k 

dldy^P[a[t,x,r])-Y,cij[t,x,r])) < c(l + |r7|r'"'+''"l^l (7.6) 
j=o 

where t e M+, x e |)7| > 1, and 

ait,x,r]]- a(oo,x, r]] e S°(E+ x r(n); if (A°''?r(n), A"-'? r(J7))) 
with r > 0. Let 

idt + nf){^e''-^'-''''''^''-iy''^)'>a[t,x,r]]^=e'^^^'''''^^^^^ (7.7) 

Then Z7(f,x, r?) G S2(l+ x r*(J7); ^(A"''? r*(J^), A"-'? r*(J7))) and 

M?,x,^)-fo(oo,x,^)eS2(l+x r(n);^(A°''?r(J7),A''''?r(n))) (7.8) 

with r > 0, where &(oo,x,}7) is a classical symbol of order 2. Moreover, we 
have 

= ^_reW^x,^)-(y,r,))^(^^^^^^_ gWoo,x,^)-(y,^))^(g^^^^^)y (7.9) 
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From Proposition 15.51 we see that for every compact set c and all 
indices a, p and iV e N, there exists Ca,p,N,K > such that 

d;^dPbit,x,^<Ca,p,NA\r]\~'' + \ri\'~''amiPit,x,r]]r), (7.10) 

where t e M+, x e |)7| > 1. Thus, 

d^dPb{<x^,x,ri) <Cas,N,K{\ri['' + \ri\~'' {\rnxl;{<x^,x,ri)f). (7.11) 

From 1 17. 8t . 1 17.101 1 and 1 17. lit , it follows that for every compact set K ci^, 
£ > and all indices a, p and NeN, there exists Ca,p,N,e,K > such that 

d;^d^'[b[t,x,r])-b{oo,x,r]))\ 

< c„,^,iv,..jc (e |"~ + 1 77 1'"" (Im , X, r^))^)) ' , (7.12) 

where ? e]R+, x e ^, |)7| > 1, r > 0. 

Choose / e C^(]R2"~^) so that xi^) = 1 when ^rj^ < 1 and /(r/) = when 
Ir^l >2. Set 

e'('^(™^-(^''')^a((X),x,r?)](l-/(r7))df]d^. (7.13) 

Put 

Six,y)= ^^J^2n-i J g'^'^^°°'''''^^"^^''^^^^(oo>^>^)^^- (7.14) 
We have the following 

Proposition 7.5. Wfe assume that Y{q] fails. Let G and S be as in [7.1 3D and 
(713 respectively. Then S + oG = I and o S = 0. 

Proof. We have 

= -J— e'W^-^''"-W)f t{t,x,rj)-i^a - 
where b[t,x, rj) is as in 117.711 . Letting f ^ oo, we get 

= - — L_^e'Woo,x,^)-(y,'?))^(oo,x,r7), 
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where b[oo,x, rj] is as in I I7.8I I and I I7.11I I. From I I7.11I I, we have 



(271) 



is smoothing. Thus n^^^ o S = 0. 



In view of II7.9I I. we have 



From Proposition l7.2[ we have 



_ ^mco.x.^)-{y.n))^^^^ r^)) (1 - x{-n))d t^dr]^ 

In view of I I7.11I I and I I7.12I I, we see that 

- e'W('»-^.'?)-(j'.'7))fo((X), X, r^)) (1 -xir]])d t^drj 

e'Wf.^'^)-(3'.'?))(^fo(f,x,r?)-Moo,x,r?)](l-;^(r?))df]dr? 





+ 

is smoothing. 
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Choose a cut-off function /i(}7)e CJ^(IR^" ^)sothat_;fi(}7)= 1 when|}7| < 
1 and Xiin) = when |r7| > 2. Take aue C^{n; A°'i T*{n]), then 

limJ(J^ ^(e'W'(^'-''''-(>'''')^a(r,x,r7))(l-/(r7))ji(^ 
-lim e'^''~y'^\l- x{r]])xi{£'n)u{y)dr]dy. 



Hence 



ThusS + ni,,^oG = 7. □ 

In the rest of this section, we recall some facts about Hilbert space the- 
ory for Our basic reference for these matters is [I]. Let A be as in 
RemarkO A has a formal adjoint A* : A"-'? T%X)) 9'[X; AO -? T*[X)), 
{A*u \v) = {u\ Av), u e C°°(X; A°'T{X)), v e C°°(X; A°'T{X)). 

Lemma 7.6. A* is well-defined as a continuous operator: 

A* : H\X; A"''? T{X)) H'+\X; A°''' r (X)) 
for all 5 e E. Moreover, we have A* = A. 

Proof The first statement is a consequence of the theorem of Calderon and 
Vaillancourt. (See Proposition 16.101 1 In view of Remark 17.41 we see that 
du^oA = I. Thus A* o d^'' = I. We have 



b 

A* - A = A* o{df oA)- A 
= iA*od^'')oA-A 
= A-A 
= 0. 

The lemma follows. □ 
From this, we get a two-sided parametrix for d^\ 

Proposition 7.7. We assume that Y{q) holds. Let A be as in Remark \7~^ 

Then d^^oA = Ao □ J?' = / . 
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Proof. In view of LemmalLl we have A*=A. Thus I = n^'oA = A*o a^^' = 

Aonf. □ 

Remark 7.8. The existence of a two-sided parametrrx for n^^^ under condi- 
tion Y[q] is a classical result. See H]. 

Definition 7.9. Suppose Q is a closed densely defined operator 

Q:HZ) DomQ ^ RanQ c H, 

where H is a Hilbert space. Suppose that Q has closed range. By the partial 
inverse of Q, we mean the bounded operator N : H ^ H such that QoN = 
n2, N oQ = TCi on DomQ, where tti, 712 are the orthogonal projections in H 
such that RauTTi = (KerQ)^, Ran 712 = RanQ. In other words, for u let 
TiiU^Qv, y e(KerQ)^p|DomQ. Then, iVu = v. 

Set DomoSf^ = { u e L\X; A'^-i r*(X)); nfu e L\X; A'^-i r*(X))}. 

Lemma 7.10. We consider as an operator 

: L\X; A°'^ T*{X)] D Domdf ^ L\X; A°-i T*[X)l 

IfY{q] holds, then has closed range. 

Proof. Suppose Uj e Domdf and dfuj = Vj ^ v in L^{X; A°-iT*{X]). We 
have to show that there exists u e Domd^^ such that d^^u = v. From 
Proposition[72l we have d^l^A = I-Fi, Ad^^ = I - Fz, where Fj, j = 1,2, are 
smoothing operators. Now, Ad^^Uj = [1- Fz)Uj Av in L\X; A°-iT*{X)). 
Since F2 is compact, there exists a subsequence Uj,^ uin L\X; A" "? T*{X)), 
A; ^ 00. We have (/ - = Av and dfuj, d^^u in H-^[X; A^-^ T*[X)), 
k^oo. Thus dfu = V. Now v e L\X; A^'iT*{X)), so u e Domdf. We 
have proved the lemma. □ 

It follows that Rann'^'^^ = (KerD^^'^^)^. Notice also that df is self-adjoint. 
Now, we can prove the following classical result 



Proposition 7.11. We assume that Y[q] holds. Then dimKerD^'^' < 00 and 
n^^^ is a smoothing operator . Let N be the partial inverse. Then N = A + F 
where A is as in Proposition \ 7. 7\ and F is a smoothing operator LetN* be the 
formal adjoint ofN, 

{N*u \v) = [u\ Nv), u e C"^{X; A"-'? r{X)], v e C"^{X; A"-'? r(X)). 

Then, N* = Non L\X; A"'"? r*(X)). 
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Proof. From Proposition |7.7[ we have A\d^^ — I—Fi, Ljf^^A — I—F2, where Fi , 
F2 are smoothing operators. Thus Ker D^^^ c Ker(/-Fi). Since Fi is compact, 
Ker(7 — Fi) is finite dimensional and contained in C°°(X; A"-'? r*(X)). Thus 
dimKer ajf^ < 00 and Kerajf ^ c C^iX; A^^'J T*iX)). 

Let {01, 02. • • • > } be an orthonormal basis for Ker The projection 
n^i) is given by n^i^u = I ^i^4'j- Thus tt^'?^ is a smoothing operator. 

Notice that / — Tr^*?^ is the orthogonal projection onto RanD^'^' since □[f^ is 
formally self- adjoint with closed range. 

For u e C°°(X; A"'-? T*{X)), we have 

[N-A)u = (AnSf^ + Fi)Nu-Au 

= Ail - 71^'? V + hNu -Au 

= -An^'^'^u + FiN{dll^A + F2)u 

= -A7i^'^'^u + Fi{I-n^'^^)Au + FiNFzU. 

Here -Att^'?), Fi(1 - 7r('?^M : H'[X; A°'i T*iX)) W+"'[X; A°'i T*iX)) V5 e R 
and m > 0, so -An^i\ Fi(l - n^^^)A are smoothing operators. Since 

Fi AAi^ : S'[X; A°''? F* (X)) ^ C°°(X; A°'^ r[X)) 

L\X; A"-'? T\X)) C^IX; A"'^ T\X)), 

we have that F1NF2 is a smoothing operator. Thus AT^ = A+F, F is a smooth- 
ing operator. 

Since Ntt^'?) = tt^'^W = = N*7r('7^ = 7r('?W* = 0, we have N* = [Nd^^ + 
Tii'i))N* = Nd^N* = N. The proposition follows. □ 

Now, we assume that Y{q) fails but that Y{q - 1), Y[q + 1) hold. In view 
of Lemma [7.10[ we see that d^^'^^ and d^^^^^ have closed range. We write 
d^H^ to denote the map: db : C^iX; A^'^i T*iX)) C°^(X; AO''?+i r*(X)). Let 
'd^l^^'* denote the formal adjoint of ^fe. We have 'd^^^^'* : C°°(X; A''''?+i F*(X)) ^ 
C°°(X; AO-'? r*(X)). Let n[''^^^ and A^^'^"^^ be the partial inverses of nSf^^^ and 
d^~^^ respectively. From Proposition !?.!!! we have 

where [N^^^^^f and (N^^'^^f are the formal adjoints of iV^'^^^^ and N^'''^^ re- 
spectively. Put 

M = ^f(Arr^^)^^l''V^;-"(Nr")^^;-^^'* (7.!5) 

and 
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In view of Proposition |7.lTl we see that M is well-defined as a continuous 
operator: M : H'iX; AP'i T*{X)) W[X; A"'"? T*[X)) and n is well-defined as 
a continuous operator: n : A''''?r*(X)) H'-'^iX; A'^'i T*{X]], for all 

sgR. 

Let 71* and M* be the formal adjoints of re and M respectively. We have 
the following 

Lemma 7.12. If we consider n and M as operators: 

n, M : A"-'? T*{X]) A"''? T* (X)), 

n* = 7i, M* = M, (7.17) 

□J?V = = 7rn!f\ (7.18) 

7r + nSf^M = / = 7r + Mn5f\ (7.19) 

ttM = = Mtt, (7.20) 

71^ = 71:. (7.21) 

Proof. From 117.151 1 and I I7.16I I. we get 1 17.171 1. 

For u e C°°(X; A0''7+i r*(X)), we have 

Thus, 

dj^ 71^ =0, 5^ 71^ =0. (7.22) 
Hence, by taking the formal adjoints 

T^h O, =0,71.^ d, =Q. (7.23) 



Similarly, 
Note that 
Now, 



□ 



^('7-1) (q-l) _ _ r-7o^, 

^^□r" = nr^r- ^r^D?"^ - d^'f- (7.25) 

^^b ^b ^^b ^^b ^^b 
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Here we used ^722) , ^72M and fT^Sl l. Hence, d^^n = 0. We have that Trajf^ 
= [d^^nf = 0, where [d^^nf is the formal adjoint of dfn. We get fTTTSl l. 
Now, 

d,''M=d':'*dr\Ni'^'W:'+^^^^^ 

= l-n. (7.26) 

Here we used CHl, ESI and fT^St . Thus, □^''^M + tt = 7. We have tt + 
Majf^ = {d^^M^nf = I, where (nSf^M+Ti)* is the formal adjoint of □Sf'M+ 
71. WegetEIHl. 
Now, 

From | |722^ . |[723l l and | |725^ . we have 

— JV^ Uj, iV^ 
_ ,.(^+l)p-('?)p<'7).* 

Similarly, we have Vf^'iV^^^ = iV^ V;"^^' V;"" Hence, 

Mil -n)= t:'\N'r'f^'r't't'X' 
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Here we used | |723^ and |[724l l. Thus, Mn = 0. We have nM = {Mnf = 0, 
where [Mn]* is the formal adjoint of Mn. We get I I7.20I I. 

Finally, n = [d^^M + n)n = tt^. We get EHJ ■ The lemma follows. □ 

Lemma 7.13. If we restrict n to L\X; A°''? T*{X)), then n = n''i\ That is, n is 
the orthogonal projection onto Kern^^^\ Thus, n is well-defined as a contin- 
uous operator: n : L\X; A«>'? T*{X)) L\X; A^'^i T*iX)). 

Proof. From I I7.18I I. we get Ranyr c KerD^'^^ in the space of distributions. 
From I I7.19I I, we get nu = u, when u e Kern^'^\ so Ranyr = KerD^'^'' and 
n^ = n = n*n = n*. For Lp, (j)eC°°[X; A°'i T*[X)),we get[[l - n)ip \ ncj)) = so 
Ran(7 — tt) X RanTi and if = [I — n)Lp + mp is the orthogonal decomposition. 
It follows that n restricted to L^(X; A" "? T*[X)) is the orthogonal projection 
onto KernJ^^ □ 

Lemma 7.14. If we consider as an unbounded operator 

uf : L\X; A°''? T\X)) D V)omuf L\X; A°-i T*{X]), 

then has closed range andM : U-{X; A°''? T*{X)) Domn|^''^ is the partial 
inverse. 

Proof From ll7J9t and Lemma Eini we see that M : L\X\ A^-'^ T*{X)) 
Domnjf^ and Rannjf' D Ran(/ - n). If ufu = v, u,v e L\X; A°-iT*iX)), 
then iI-n)v = U- n]dl^^u = v since Trnjf^ = nSfV = 0. Hence Rannjf' c 
Ran (7 - tt) so has closed range. 

From 1 17.201 1, we know that Mn = nM = 0. Thus, M is the partial inverse. 

□ 

From Lemma [7. isl and Lemma [7.14| we get the following classical result 

Proposition 7.15. We assume that Y[q) fails but that Y[q - 1) and Y{q + 1) 
hold. Then d^'' has closed range. Let M and n be as in (7.1Si) and [7.16i) 
respectively. Then M is the partial inverse of d^^ and n — n^''\ 

8 The Szego Projection 

In this section, we assume that Yiq] fails. From Proposition 17.51 we know 
that, for every local coordinate patch Xj , there exist 

Gj e L-MXj; A"'-? riXjlA^'" r(Xy)), Sj e Ll ,iXj; A°''? r(Xy), A"''? T%Xj)) 
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such that 



Sj+nfGj = i 



(8.1) 



in the space Q'[Xj x Xj; 5^{A°-'' T*[Xj),K^'^ T*{Xj))). Furthermore, the distri- 
bution kernel Kg^ of Sj is of the form 



(8.2) 



where xp{oo,x,r]), a{oo,x,r}) are as in Proposition |0] and 1 17.41 1. From now 
on, we assume that Sj and Gj are properly supported operators. 

I \k 

We assume that X — {jj^iXj. Let Xj be a C°° partition of unity subordi- 
nate to {^j}- From I IB.ll l. we have 



in the space Q}'{Xj x Xf, if (A"-'? T\Xj),AP'i T\Xj))). Thus, 

\ □Sf^s = o 

in X X; if (A"''? r*(X), A"-'? T*{X))), where 

S, G : A"-'? T* (X)) ^ A"''? T* (X)), 
it 

= ^ ( u),ue A°''? r (X)) 
j=i 

k 

Gu =^Gj{XjU),u&^'{X] A^-TiX)) 



(8.3) 



(8.4) 



(8.5) 



From 1 18.41 1. we can repeat the method of Beals and Greiner (see page 173- 
page 176 of 111) with minor change to show that S is the Szego projection 
(up to some smoothing operators) if \J^I^^ has closed range. 

Let S*, G* : ®'(X; A°''JT*iX)) Q'[X; K^-'fT*[X)) be the formal adjoints 
of S and G respectively. As in Lemma [7!6l we see that S* and G* are well- 
defined as continuous operators 



' S* : H\X; A"-'? T{X)) H'{X; A°-i T%X)) 
G* : H'{X; A°''? T*{X)) H'+\X; A"-'? T* (X)) ' 



(8.6) 



for all 5 e E. We have the following 
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Lemma 8.1. LetS he as in I8.4\i , f8.5D . We have S = S*S. It follows thatS = S* 
andS^ = S. 

Proof. From it follows that S* + G*df = I. We have S = (S* + G*df) o 
S = S*S + G*nSf = S*S. The lemma follows. □ 

Let 

H = iI-S]oG. (8.7) 
H is well-defined as a continuous operator 

H : H'{X; A°''? T*{X)) H'+\X; A°''? r(X)) 

for all 5 e E. The formal adjoint H* is well-defined as a continuous opera- 
tor: H* : H'iX; A"-'? T*iX)) H'+^{X; A°-'J T*iX]), for all s eR. 

Lemma 8.2. LetS and H he as in WAD , {8.3i) and W.TO . Then 

SH = 0, (8.8) 
S + dfH = I. (8.9) 

Proof We have SH = S{I - S)G = {S- S^)G = since = S, where G is as in 
D . From D , it follows that S + d^H = S + d^\l -S)G = I- dfSG = I. 
The lemma follows. □ 

Lemma 8.3. LetH be as in ffi./D . Then H = H*. 

Proof. Taking the adjoint in 1 18.91 1, we get S* + H*d^'' = /. Hence 

H = iS* + H*df)H = S*H + Wd^l^H. 

From LemmaO LemmaO we have S*H = SH = 0. Hence H = H*dfH = 
H*. □ 

Summing up, we get the following 

Proposition 8.4. We assume that Y[q) fails. Let 

j S : ¥[X; A°''' r[X)) ®'(X; A°''' T*[X)) 
\ H : ®'(X; A"-'? T* (X)) ^ ®'(X; A"''? T* (X)) 

be as in (8.50 and (8.7V . Then, S and H are well-defined as continuous oper- 
ators 

S : H'[X; A"'*? r[X]] H'iX; A"'*? r(X)), (8.10) 
H : H'{X; A"-'? T*{X)) H'+\X; A"'"? T{X)), (8.11) 
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for all 5 e M. Moreover, we have 

Hd^^ + S = S + dfH=I, (8.12) 

dfs = sdf = 0, (8.13) 

S = S* = S\ (8.14) 

SH = HS = 0, (8.15) 

H = H*. (8.16) 

Remark 8.5. If S', H' : A"-'? r*(X)) ^ AO'^r*(X)) satisfy <8J0t - 
llSia , then S' = {Hd^^ + S)S' = SS' = Sid^H' + S'] = S and 

H' = [Hd^^ + S]H' = {Hd^^ + S')H' = Hd^^H' = Hid^^H' + S') = H. 

Thus, I I8.10I I - I I8.16I I determine S and H uniquely up to smoothing operators. 
Now we can prove the following 

Proposition 8.6. We assume that Y[q) fails. Suppose d^^ has closed range. 
Let N he the partial inverse of d^^ . Then N = H+ F and tt^*?^ = S + K, where 
H, S are as in Proposition \8.4[ F, K are smoothing operators. 

Proof We may replace S by I-d^H and we have d^H+S = / = H*df+S*. 
Now, 

= n^i\d;i^H + S) = n'-'^^S, (8.17) 

hence 

[n^^))* = S*(7r('?))* = Trf-?) = 5*71^'?'. (8.18) 



Similarly, 



S = [Nd^^ + 7r('?^)S = n^'i^S + iVFj, (8.19) 



where Fi is a smoothing operator. From 1 18.171 1 and 1 18.191 1. we have 

S - n^i'> = S- n^'^^S = NFi . (8.20) 
Hence (S* - 7r('?^)(S - n^i^) = F*N^Fi. On the other hand, 

(S* - n^^^lS - n^^^] = S*S - S*n^'^^ - n^^^S + {n^'^^f 

= S-n^''^ + Fz, 

where F2 is a smoothing operator. Here we used 1 18.171 1 and J8.18t . Now, 

F^iV^Fi : A°''? T*{X)] C^iX; A"-'? T*{X]) 

L\X; A"-'? r (X)) ^ C^iX; A"''? T%X]). 
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Hence F*N^Fi is smoothing. Thus S — n^^^ is smoothing. 
We have, 

N-H = Nidll^H + S)-H 

= (/-7r''?')H + ArS-H 
^NS-7i^''^H 

= NF4 + F3 

where F4 and F^ are smoothing operators. Now, 

N-H* = N*-H* 
= F*N + F* 

= F*{NFi + Fi + H] + F;. 

Note that 

F^NFi : A°''? r (X)) ^ C°°(X; A"'"? r (X)) 

^ L\X; A"'-? r (X)) ^ C^CX; A"''? T* (X)). 

and F;H : H^(X ; A"''? r*(X)) -> H^+^CX ; A"'-? r*(X)) for all 5 e E and m > 0. 
Hence N— H* is smoothing and so is [N — H*)* = N — H. □ 

From Proposition l8.4l and Proposition l8.6[ we obtain the following 

Theorem 8.7. We recall that we work with the assumption that Y{q) fails. 
Let[n-, n+), ri- + n+ = n — I, be the signature of the Levi form L. Suppose 
D^^^ has closed range. Then for every local coordinate patch U c X, the dis- 
tribution kernel of n^^^ onUx U is of the form 

KMx,y) = ^^^^ J e'^^^'^-''-^^-(y-^)^a[oo,x, r]]dr], (8.21) 

a((X), X, r]] e S%iT*iU) ;J^iA°'i T*[U), A"-'? r*(C7))), 

00 

fl(oo, X, 17) ^ flj(oo, X, rj) 


in the Hormander symbol space S%[T*[U)] i^' (A"-'? r*(C/),A0 '7r*( [/))), where 
aj{(X),x, ri) e C^mU); if (A"''? T*{U], A"-'? T*iU])], j=0,l,..., a^Coo, x, Ary) = 
X~j aj[oo,x,ri), A > 1, |r7| > 1, 7 = 0,1,.... Here xp {00, x,rj) is as in Propo- 
sition l4^ and (4.80 . We recall that ip {00, x,ri] e C°°(r*([/)), ijj[oo,x,Xri) — 
/lip [00, x,ri), A > 0, Ivaip {00, x,ri) >^\ri\^ (dist((x, and 

xp{oo,x,ri] = —ip{oo,x,—ri). (8.22) 
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Moreover, for all j = 0,1,..., 

Ujioo, X, 17) = in a conic neighborhood of E+, ii q = n-, n- / n+, 
aj{oo,x,ri) = in a conic neighborhood of T.~ , if q = n+, n-^n+. 

(8.23) 

In the rest of this section, we will study the singularities of the distribu- 
tion kernel of the Szego projection. We need 

Definition 8.8. Let M be a real paracompact C°° manifold and let A be a 
C°° closed submanifold of M. Let U be an open set in M, U{^k / 0. We 
let C^([/) denote the set of equivalence classes of / e C^iJJ) under the 
equivalence relation 

f = g in the space C^([/) 

if for every zq e [/, there exists a neighborhood W a U oi Zq such that 
f = g + h on W, where h e C°°(l/K) and h vanishes to infinite order on 

aHvk- 

In view of Proposition 14.31 we see that ip[c)0,x, rj) has a uniquely deter- 
mined Taylor expansion at each point of E. Thus, we can define ijj{oo,x, rj) 
as an element in C^{T*{X)). We also write ip{oo, x,rj) for the equivalence 
class of ip{oo,x, rj) in the space C^{T*[X)). 

Let M be a real paracompact manifold and let A be a closed sub- 
manifold of M. If Xq e A, we letyl(A, n,Xo) be the set 

A{A, n,Xo) = {{U,fi,...,f„y, Uis an open neighborhood of Xq, 

/i,...,/„eC^(C/),/j|Ap[7 = 0, j = l,...,n,dfi,...,dfn 

are linearly independent over C at each point of U]. (8.24) 

Definition 8.9. If Xq e A, we let A^coiK n,Xo) denote the set of equivalence 
classes of A(A, n,Xo) under the equivalence relation 

Fi = ([/, /i, ...,/„) ~ Fa = (V, gi, ... , g„), Fi,F2 G A{A, n,Xo), 

if there exists an open set W c Uf] V of Xq such that gj = Xfc=i ^j.kfk in 
the space C^iW), j = l,...,n, where aj,k e C^iW], j, k = l,...,n, and 
[(^j,k)j is invertible. 

If (C/,/i, . . .,fn)eA[A, n,Xo),we write . . . ,/„)xo for the equivalence 
class of [U,f\,...,fn) in the set A;c„(A, n,Xo), which is called the germ of 
(C/,/i,...,/„)atXo. 
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Definition 8.10. Let M be a real paracompact C°° manifold and let A be a 
closed submanifold of M. A formal manifold of codimensin A; at A 
associated to M is given by: 

For each point of x e A, we assign a germ r^^ e ^x(A, A;, x) in such a way that 
for every point Xq e A has an open neighborhood U such that there exist 
/i, . . . , /fc e C^(C/), fj\Af~^u = 0,j = l,...,k,dfi,...,dfk are linearly 
independent over C at each point of U, having the following property: 
whatever x&U, the germ [U,fi,...,fk)xis equal to Tjc. 

Formally, we write — {F^; x e A}. If the codimension of is 1, we call a 
formal hypersurface at A. 

Let n = [Tx] X e A} and Jli = ^Tx', x e a| be two formal manifolds at A. If 
Fx = Fx, for all x e A, we write Jl = fJi at A. 

Definition 8.11. Let O = {Fx; x G A} be a formal manifold of codimensin k 
at A associated to M, where A and M are as above. The tangent space of 
at Xo e A is given by: 

the tangent space of f2 atxo = eCTxoiM); ^<i/j(xo), = 0, ; = 1, . . . , fcj , 

where CTx^M) is the complexified tangent space of M at Xq, {U,f\,...,fk) 
is a representative of Fx^. We write T^JS^) to denote the tangent space of Jl 
atxo. 

Let (x, y) be some coordinates of X x X. From now on, we use the nota- 
tions E, and rj for the dual variables of x and y respectively. 

Remark8.l2. For each point (xo,}7o.->!^o.)7o)e diag(I! x E), we assign a germ 

r(x„,r7„,x„,r7„) = (r*(X) X T* (X), ^ - lp'^{oO,X, ri),y - Xp'^{(X>, X, )7))(xo,r7o,xo,r;o)- (8-25) 

Let Coo be the formal manifold at diag(I! x S): 

Coo = {F(x,,7,x,r)); (x, Y], X, 17) G diagCS X S)} . 
Coo is strictly positive in the sense that 

-Mv,v)>Q, yve rp(Coo)\Crp(diag(E x E)), 

wherep ediag(SxE). Hereaisthe canonical two form onCT^(X)xCT^(X) 
(see CT ). 

The following is well-known (see section 1 of ||30], for the proof) 
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Proposition 8.13. There exists a formal manifold J+ = (x,?7)eE| 
at E associated to T*{X) such that codim/+ — n - I and V(xo,J7o) e T., if 
{U,fi,. . .,fn-i) is a representative ofj^xo.m)' ^^^^ 

{fjjk} = in the space C^{U), j,k^l,...,n-l, (8.26) 

n-l 

Po=^gjfj in the space C^{U), (8.27) 
where gj e C^{U], j = l,...,n-l, and 

yC7(///.,Hy )>0 at ixo,iqo)eJ:, j^l,...,n-l. (8.28) 
We also write fj to denote an almost analytic extension offj . Then, 

fj{x,ijj'^[oo,x,ri]) vanishes to infinite order on T., j = l,...,n-l. (8.29) 
Moreover, we have 

n—\ n—l 

TpiCoo) ={(z^ +2 fj^fMo, r]o), v + '^SjHjixo, r]o)]; 

7=1 7=1 

i; e 7(^„,^„)(S), tj,SjeC,j = l,...,n-l], (8.30) 

where p = [xq, r]o,Xo, rjo) e diag(S x S) and Coo is as in Remark WT^ 

We return to our problem. We need the following 
Lemma 8.14. We have 

xP"^^[^,p,OJo[p))o}Q[p) = (8.31) 

and 

Rank (j/^'^Voo, p, Wo(p))) =2n-2, (8.32) 

for all pGX. 

Proof. Since ip'^[oo,x,rj) is positively homogeneous of degree 0, it follows 
that ip'^^ioo,p,Ci>oip))cooip) = 0. Thus, Rank^^ip'^^ioo, p, cooip)fj <2n-2. 
Fromlmip{oo,x,ri]>: |r7| dist((x, j^); Hf, wehavelmijj'^^ioo,p,coo{p))V ^ 
0,ifV^ {Xcooip); A e C}. Thus, for all V ^ {Xcooip); A e C}, we have 

(^iP';^^{oo,p,a)oip]]V,v) = (Reip';^^[oo,p,a)o{p))V,v) 

+ i(lmip';^^{oo,p,coo[p]]V,v)y^O. 

We get {833. □ 
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Until further notice, we assume that q = n+. For p e X,we take local 
coordinates x = (xi, X2, . . . , Xzn-i) defined on some neighborhood ^ofpGX 
such that 

coo[p) = dx2n-\, x{p) = (8.33) 

and A"'! Tp[X)®A^-°Tp[X) = {Xjj^ aj^C, j^l,...,2n- 2}. We take 

n so that if Xo e then 170,2^-1 > where WoUo) = (J7o,i, ■ ■ ■ , J7o,2«-i)- 

Until further notice, we work in Q and we work with the local coordi- 
nates X. Choose /(x, 17) e C°°{T*[X]) so that /(x, 17) = 1 in a conic neighbor- 
hood of {p,cooip]], x[x,r]) = outside T*(fl], /(x,)7) = in a conic neigh- 
borhood of E" and xix,A.ri) — /(x,}7) when A > 0. We introduce the cut- 
off functions xi^'l) (1 ~ ZC-^.'?)) in the integral I I8.21I I: K„ic,]{x,y) = 
K^Mix,y) + K^Mix,y), 

-„w)(x,y)= J e'^'^^'^-'-^^-(y-^)\l-x{x,r]]Moc,x,r])dr]. (8.34) 



K 



Now, we study K^u,]. We write t to denote r/z^-i- Put 17' = (171, . . . , r]2„-2)- We 
have 



(271 



(271) 



^U(^(co.x,(w,l]]-{y.(w,l)))^2n-2^^^^^^^^^ t))a{00,X,{t W, t))dwdt 



^ 



(8.35) 



where rj' = tw, w E M^^-z xhe stationary phase method of Melin and Sjos- 
trand (see Proposition |A.14| l then permits us to carry out the w integration 
in llOSll . to get 



• 00 

Kj,ix,y)= I e'"t'^^'''yh+{x,y,t)dt (8.36) 
^ Jo 

with 

00 

s+{x,y,t)r^Y,^i{x,y]t"-'-i (8.37) 



in S1-\n X J7x]0,oo[; if (A"-'? r*(X), A"-'? T*{X])], where 



5i(x,y) e C°°(J7 X n; if (A°''? r(X), A"-'? r(X))), 7 = 0, 1, ... , 
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and (p+{x,y] e C°°(r2 x n] is the corresponding critical value. (See Proposi- 
tion |A.13| for a review.) For x e Jl, let (7{x] e E^^-a i-,g ^j-^g vector: 

(x,((7(x),1))gS+. (8.38) 

Since dw[ip{c)0,x,{if, 1))- (y.C"^. 1))) = at x = y, w = cr(x), it follows that 
when x—y, the corresponding critical point is u; = cr(x) and consequently 

0+(x,x) = O, (8.39) 
(0+);(x, x) = ip'^ioo,x,{cr{xl 1)) = (cr(x), 1), (0+);(x,x) = -(cr(x), 1). (8.40) 

The following is well-known (see Proposition |A.13l l 
Proposition 8.15. In some open neighborhood Q ofp in f2, we have 

Im0+(x,j/)> c inf (lmip{oo,x,iw,l]] + \du;[ipioo,x,{w,l))- {y,iw,l)))\], 
(x,y)eQxQ, (8.41) 

where c is a positive constant and W is some open set of the origin in R^"~^. 

We have the following 

Proposition 8.16. In some open neighborhood Qofp in il, there is a con- 
stant c> such that 

Im^+(x,y)> c|x'-y'|^ {x,y]eQxQ, (8.42) 
where x' = [xi, X2„-2), y' = iyi, ■■■ ,y2n-2) and\x' - y'\^ = [xi - yif -\ h 

iX2n-2 ~y2n-2T- 

Proof. From i/'((X),x,(iA', 1))- (y,(u;,l)) = (x -y,(if , 1)) + 0(|u; - cr(x)|^) we 
can check that 

duj{ip{oo,x,iuj,l]]-{y,{u','i^)))={x'-y',dw)+0{\w-a{x]\), 

where (7(x) is as in 1 18.381 1 and x' = (xi,...,X2„-2), y' — (yi,---,y2n-2)- Thus, 
there are constants Ci, C2 > such that 

\du;{ip{oo,x,iw,l]]- {y,{w,l))]f > ci \x' - yf - C2\w - a{x)f 

for (x,u;) in some compact set of X M2«-2 jf a (;^;'_yQ > C2 |if - cr(x)| , 
then 

\do,ixPioo,x,co)-{y,co)f > y |(x'-yO|'. (8.43) 
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Now, we assume that |(x' - y')^ <^\w- cr{x)f. We have 

1111^(00, x,{w, l))>C3\w- a{xf > ^ |(x' - y'f , (8.44) 

2c2 ' ' 

for (x, w] in some compact set of ri x R^"-^, where C3 is a positive constant. 
From I I8.43I I. II8.44I I and Proposition l8.15[ we have lm(p+{x,y) > c — }/')| 
for X, y in some neighborhood of p, where c is a positive constant. We get 
the proposition. □ 

Remark 8.17. For each point (xo,Xo) e diag(Jl x rj), we assign a germ 

H+,ix,.x„] = inxrt, cj)+{x,y)\x,,x,). 
Let H+ be the formal hypersurface at diag(J7 x fi): 

H+ = (x,x) e diag(J^ x n)} . (8.45) 

The formal conic conormal bundle A^^f of H+ is given by: For each point 

[xo,r]o,Xo,r]o)edmg[{J:+f]rm x (S+p| nmi 
we assign a germ 

A(xo,^o,x„,,„) = (r ([/) X rm - (0+);^. ; = 1, . . . ,2^ - 1, 

r]k-i(p+yyj, k = l,...,2n-2, 0+(x,y))(:co,f;o,xo,r;„), 



where t — ' and C7 c r2 is an open set of Xo such that {(b+)' 7^ on 
UxU. Then, 

A0,f = {A(x,,,x,^); (x, ^,x,r7) e diag((S+p| r(J^)) x (S+ fj r (J^)))} . (8.46) 

A^^f is a formal manifold at diag((E+n x (S+fl In fact, A^^t 

is the positive Lagrangean manifold associated to cj)+t in the sense of Melin 
and Sjostrand. (See [29] and Appendix A.) 
Let 

[W,fi{x, E,,y, rj), . . .Jin-zix, ?,y, }?)) 
be a representative of r(;,„,^„,;,„,^„), where r(;,„,^„,;,„,^„) is as in I I8.25L Put 

Let be the formal manifold at diag(E+ x 

C = {r[x,,,x.,); ^' ^' ^) e diag(E-^ X . (8.47) 
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We notice that ?/'(oo,x,)7)-(y, J?) and ^+(x,y)f are equivalent at each point 
of 

in the sense of Melin-Sjostrand (see Definition |A.19| l. From the global the- 
ory of Fourier integral operators (see Proposition lA.20l l, we get 

A^,, = C'^ at diag((S+ f| rm x (E+ f] rmi (8.48) 

See Proposition lA.GI and Proposition lA.20[ for the details. Formally, 

Coo = {{x, ^,y, rj); {x, ^,y, -17) e A^^f } . 

Put (p+{x,y) = -cj)^[y,x). We claim that 

A^,, = A^^, at diag((E+ f) r{n]] x (S+ f] rmi (8.49) 

where A^^^ is defined as in I I8.46I I. From Proposition 14.51 it follows that 
^+(x,y)f and-^_,_(y,x)f are equivalent at each point of diag((S+P| r*(n))x 
(E+ Pi T*(SYj)) in the sense of Melin-Sjostrand. Again from the global theory 
of Fourier integral operators we get I I8.49I I. 
From I I8.49D . we get the following 

Proposition 8.18. There is a function f e C^i^l x n), f{x, x) 7^ 0, such that 

0+(x,y) + /(x,y)^+(y,x) (8.50) 
vanishes to infinite order onx = y. 

From 1 18.501 1, we can replace 0+(x,y) by |(0+(x,y)- 0^(y,x)). Thus, we 
have 

0+U,y) = -0+(y,-^). (8.51) 

From 1 18 .401 1, we see that [x,dx4>+{x,x])eT.+, dycj)+{x,x) = -dx(j)+ix,x). We 
can replace 0+(x,y) by ||,^,,,.,^,tHg^,(,,,)|| • Thus, 

(ix0+(x,x) = C(Jo(x), dy(j)+[x,x) = -coo[x). (8.52) 

Similarly, 

e'^-i--y^'s4x,y,t)dt, 



where K^ici]{x,y) is as in I I8.34I I. From I I8.22I I, it follows that when q — n- — 
n+, we can take 0_(x,y) so that 0+(x,y) = -0_(x,y). 
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Our method above only works locally. From above, we know that there 
exist open sets Xj,j = l,2,...,k,X = [jj^^ Xj , such that 

J> 00 


on xX^-, where (p+j satisfies ll09ll , iImD- iIMSI, llOSt . lIS^H . JHSSI and 
s+j[x,y, t), j = 0, 1, . . . , are as in I I8.37I I. From the global theory of Fourier 
integreil operators, we have 

^^..f = C = at diag((E+p| r(X; f]X,]) x (E+p| r(X; f|X,))), 

for all j,k, where A^^^ f , A^^^t are defined as in II8.46I I and is as in II8.47II . 
Thus, there is a function fj^k e C°°((Xj fl^-t) x (^j fl^fc))' such that 

(p+jix,y) - fj,kix,y)(j)+,kix,y) (8.53) 

vanishes to infinite order on x = y , for all j,k. Let Xj[x,y) be a C°° partition 
of unity subordinate to |Xj x X/j with jj(x,y) = /j(y,x) and set 

From 1 18.531 1 and the global theory of Fourier integral operators, it follows 
that 0+j(x,y)f and 4>+{x,y)t are equivalent at each point of 

diag((E+p| r(X,)) X (E+pl r(X,))) 

in the sense of Melin-Sjostrand, for all Again, from the global theory of 
Fourier integral operators, we get the main result of this work 

Theorem 8.19. We assume that the Levi form has signature n+), n- + 
n+ — n — l. Letq = U- or n+. Suppose\li^^ has closed range. Then, 

Kj,{c,)-K^{^ if n+ = q^n-., 
K^{q] — K^u,] if n- — q^ n+, 
K^{q] — K^{q) + K_^{q) If n+ — q — U-, 

where K (q]{x,y) satisfies 

J> oo 
e"f-^'-y^'s+lx,y,t)dt 


with 

s+{x,y, t) e Sl-\X X Xx]0, oo[; ^(A°''? r(X), A"'-? r{X))), 



75 



s+ix,y,t]-^^si{x,y)t"-'-j (8.54) 



inSl-\XxXx]0,(X)[; ^(A0''7r*(X),A«-'?r*(X))), where 

si{x,y] e C^iX X X; if (A"-'? r(X), A"-'? r(X))), 7 = 0, 1, ... , 

0+(x,y)eC°°(XxX), Im(/)+(x,y)>0, 

0+(x,x) = O, 0+(x,y)7^O ifx^y, 
dx4>+7^0, dy4>+7^0 where lm(f)+ = 0, 
dxcj)+ix,y)\jc=y = coo[x), dyCj)+[x,y)U=y = -coo[x], (8.55) 
0+(x,y) = -^+(y,x). 
Moreover, cj)+{x,y) satisfies i8.4W . Similarly, 

J> 00 


with 



S-[x,y,t)-^Yj^L[x,y)t" 



inSl^o [XxXx]0,oo[; £e{A°-'^T*[XlA°-'iT*[X))l where 



sUx,y] e C^iX X X; if (A"''? r(X), A"''? r(X))), 7 = 0, 1, ... , 
and when q = n = n+, 0_(x,y) = -0_^(x,y ). 

9 The leading term of the Szego Projection 

To compute the leading term of the Szego projection, we have to know the 
tangential Hessian of 0+ at each point of diag(X x X) (see 1 19. Il l), where 0+ 
is as in Theorem 18.191 We work with local coordinates x = (xi,...,X2„-i) 
defined on an open set f2 c X. The tangential Hessian of 0+(x,y ) at {p, p) G 
diag(X X X) is the bilinear map: 

Tip,p]H+ X T[p^p)H+ C, 

iu,v)^ {i(p'_pip,p)u,v) , u,v eT[p_p)H+, (9.1) 



where H+ is as in 1 18.451 1 and (0+)'' 



(0+):x (0+);v 
(0+);, (0+);; 
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From I I8.55I I, we can check that T(^p^p)H+ is spanned by 

{u,v), {Y{plY{p)), u,f;eAi'''rp(X)©A°'irp(X). (9.2) 

Now, we compute the the tangential Hessian of (p+ at {p, p) e diag(X x 
X). We need to understand the tangent space of the formal manifold 

at p = {p, Acoo(p)) e S+, A > 0, where /+ is as in Proposition |8.131 

Let Xj, i — l,...,n — \,he the eigenvalues of the Levi form Lp. We recall 
that 2/ |Aj| |cr,j.(p)|, j = \,...,n-l and -2i |Aj| |cr;7(p)|, ; = 1, . . . , n - 1, are 
the non- vanishing eigenvalues of the fundamental matrix i),. (See I I3.11I I.) 
Let A+ c CTp{T*{X)) be the span of the eigenspaces of Fp corresponding to 
2i I Aj I |(7;y(p)|, = 1, . . . , n - L It is well known (see [32], ||30] and Boutet de 
Monvel-Guillemin [lOl) that 

rp(/+) = crp(E)®A+ k+^TpU+Y, 

where Tp (/+)-■- is the orthogonal to Tp{J+) in CTp{T*{X)) with respect to the 
canonical two form cr. We need the following 

Lemma 9.1. Let p = {p,Xcoo[p)) e S+, A > 0. Let Zi{x),...,Z„-i{x) bean 
orthonormal frame o/A^'°r^(X) varying smoothly withx in a neighborhood 
of p , for which the Levi form is diagonalized at p . Letqjix,^), j = l,...,n-l, 
be the principal symbols of Zj{x), j — \,...,n — \. Then, A+ is spanned by 

r H,^[p),ii ]\qj,qMp)>Q 
\ H^^iplif '-{qj,qj}ip]<0 ■ 

We recall that (see HM ) y [qj,qj]ip) = -2XLp(Zj,Zj). 

Proof In view of I IS.IOI I. we see that Hqj[p] and Hq.[p) are the eigenvectors 
of the fundamental matrix Fp corresponding to | qj , qj j (p ) and ^qj , qj | (p ), 
for all j. Since A+ is the span of the eigenspaces of the fundamental matrix 
Fp corresponding to 2/A|Aj|, j = l,...,n - 1, where Xj, j = l,...,n - 1, are 
the eigenvalues of the Levi form Lp. Thus, A+ is spanned by 



^(P), if 7 
Hq^ipl if 7 



□ 
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We assume that {U,fi,..., fn-i) is a representative of Jp . We also write fj 
to denote an almost analytic extension of fj, for all j. It is well known that 
(see EQl and there exist hj{x,y) e C°°(X xX), j = l,...,n-l, such 

that 

fj{x,icl)+l)-hjix,y)cl)+ix,y) (9.4) 

vanishes to infinite order onx = y,7 = l,...,n — 1. From Lemma [977] we 
may assume that 



Hf^{p)^H,.{pl if i 
[ Hf.{p]^H^^ip), if I 



qi,q,\ip] > 



Here qj, j — l,...,n - 1, are as in Lemma lSTTl 
We take local coordinates 

X = {xi,... ,X2n-i)> Zj =X2j-i + ix2j, j = l,...,n — l, 

defined on some neighborhood of p such that cooip) = \/2dx2n-i, x[p) = 0, 
^£-^P^ I ^fc^/^)) = 7, A; = 1, . . . , 2n - 1 and 

where Zj, j = l,...,n-l, are as in Lemma [97n cj & C, — — ^'^)> 

j — l,...,n — I and A^, ; = 1, . . . , n — 1, are the eigenvalues of Lp. (This is 
always possible. See page 157-page 160 of [IJ.) 

We may assume that > 0, j = l,...,q, Xj <0, j = q + l,...,n - I. Let 
^ = (<^i, . . . , ^2n-i) denote the dual variables of x. From 1 19.51 1. we can check 
that 

i 1 _ 1 I |2 

fjix, Q = --(?2;-l - i^2j] - -j^iZi^2n-\ + i-^CjX2n-\^2n-\ + 0(|(X, f)] ), 

j = l,...,q, ^' = i^i,...,^2n-2), 

i 1 1 _ I , |2 

/iU-?)= -(?2i-l + Z'^2i)-^AyZj?2«-l- Z^C;X2„-li^2«-l + 0(|(X,<^ )| ), 

;=^7+l,...,n-l, f = (?!,..., ^2n-2). (9.6) 

We write y = (yi, . . . ,y2„-i), wj = y2j-i + iy2j, j = l,...,n-l,^ = ^(^ - 
^'4^' 4 = K^:: + '-4^' 7 = 1--" - 1 and 4 = 1(4:^'+ i^lj = 
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1, . . . , n — 1. From I I9.4I I and I I9.6I I, we have 

^0+ 1 — ^4'+ I |2 

-/^ -=i^.jZj- iCjX2n-iH = /i;0+ + O( (x,y) ), j = l,...,q, 

OZj V2 £7X2^-1 

1 — ^0+ I |2 

-j^i^i^i + iCjX2n-i)^ = hj(l)+ + 0[\{x,y]\ ), 

OZj V2 OX2n-l 

j = q+l,...,n-l. (9.7) 
From I I9.7I I. it is straight forward to see that 

d^(h+ ^20 ^20 



dzjdzk dzjdzk dzjdwk dzjdwk 

d^(h+ ^20 

-(0,0)+ - 7 (0,0)-c,=0, 



l<i<q, l<k<n-l, (9.8) 



and 



d^Cj)+ ^20 ^20 ^20 

^^1^(0, 0) = ^^^(0, 0) + i^j 5j,k = ^^p-(0, 0) = :1 (0, 0) 

d^(h+ ^20 
= ^- / (0, 0) + 7 (0, 0) - Cj = 0, 

dZjdx2n-i dZjdy2n-i 

q + l<i <n-l, l<k<n-\. (9.9) 

Since d^(p+\^=y = cooix], we have /^.(x,(0+)^(x,x)) = 0, ; = - 1. 

Thus, 

^0+ 1 _ ^0+ ? 
/— ^(x,x)-— (AjZj + ;cyX2„-i)^ (x,x) = 0(|x| ), j = l,...,q, 

OZj V2 OX2n-l 

- i^J^ix,x]--^iA.jZj - iCjX2n-i) f^^ (x,x) = 0(|x|^), 

dZj V2 dX2n-l 

j = q + l,...,n-l. (9.10) 
From 1 19. lot , it is straight forward to see that 

^20, ^20 ^20 ^20 

^^1^(0, 0) + ^_ :1 (0, 0) = ^^^(0, 0) + (0, 0) + i A,- 5j,k 

OZjOZk OZjOWk OZjOZk OZjOWk 

d^(b+ ^20 
= ^- 7 (0, 0) + 7^ (0, 0) - -Cj = 0, 

dZjdX2n-l dZjdy2n-l 

l<i<q, l<k<n-l, (9.11) 
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and 



d^ch+ d^d)+ d^(i)+ d^d)+ 

-(0, 0) + , (0, 0) = , V_ (0, 0) + , (0, 0) - iXj 5j,k 



dzjdzk dzjdwk dzjdzk dzjdwk 

= ^ 7 (0, 0) + , 7 (0, 0) - Cj = 0, 
q + l<i <n-l, l<k<n-\. (9.12) 
Since (p+{x,y) = -^^(y,x), from II9.9I I. we have 

-(0,0) = -— ^(0,0) 



dZjdiVk dwjdzk 



dWjdzic 

Combining this with I I9.11I I, we get 



^(0,0) = 0, q + l<k<n-l. 



dZjdZk 

Similarly, 



(0,0) = 0, \<i<q, q + \<k<n-\. (9.13) 



dZjdZk 



(0,0) = 0, l<i<q, l<k<q, 

(0,0) = 0, q + \<j <n-\, q + \<k<n-\. (9.14) 



dzjdzt 

From 1 19.81 1 and 1 19.111 1, we have 

{<d,Q)^-iXj5j,k-—^{Q,0) = -2iXj5j,k, l<j,k<q. (9.15) 



dZjdWk dZjdZk 
Similarly, 



3"^ (b 

[0,0] = 2i?.j5j,k, q + l<j,k<n-l. (9.16) 



dZjdwk 
Since = 0, we have 

^ (0,0) + 2- (0,0) + - ^ (0,0) = 0. (9.17) 

O X2n-lC X2n-1 O X2n-l<^ Yln-l O y2n-l<^ yin-l 

Combining ||9JT] |, ll9J2l l, JgT^t . JCTt . ll9J5l l, lEH and ll9T7ll . 

we completely determine the tangenticd Hessian of 0+(x,y ) at [p, p). 
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Theorem 9.2. With the notations used before, in some small neighborhood 
ofip, p)eXx X, we have 

n—l n—\ 
0+(x,y) = ^/2{X2n-l-y2n-l)+i^\^i\\zj - W jf +^[iXj{ZjWj-ZjWj) 



+ Cj{ZjX2n-l ~ U!jy2n-l) + Cj{ZjX2n-l — U) jy2n-l)^ 

+ ^/2{x2n~-l- y2n-i)f{x,y) + 0{\{x,yi\), 

feC^, /(0,0) = 0, f{x,y] = fiy,x). (9.18) 
We have the classical formulas 

r°° _^ j m\x-"'-\ ifmeZ,m>0 

Jo ' '''"'^'^1 (^x—Hlogx + c-Xr"'})' ifmeZ,m<0 " 

(9.19) 

Here x 7^ 0, Rex > and c is the Euler constant, i.e. c = limm_oo(Xr J ~ 
logm). Note that 



eit>^'-^'y^'Y,4ix,y)t"-'-^dt 

'0 j=o 



• 00 



= lim V5i(x,y)r-i-^'d/:. (9.20) 

Jo j=0 

We have the following corollary of Theorem l8.19l 
Corollary 9.3. There exist 

F+, G+, F-,G-e C^iX X X; ^(A°'^ T* (X), A°'^ T*{X))) 

such that 

K^i,, = F+(-/((/)+(x,y)+ /O))-" +G+log(-«((/)+(x,y) + /O)), 

K^in^ = F_(-/(</)-(x,y)+ /O))-" + G_log(-/(<^_(x,y) + «0)). 
Moreover, we have 



n-l 



F+ - 1 - k)\sl{x,y){-ict)+{x,y)f+U{x,y\(t>+{x,y)T, 



n-l 

F. ^YSn - 1 - k)\st{x,y){-i(t>-{x,y)f + f.{x,y){cl).{x,y)y\ 



G+=2]^-^^5f'(x,y)(-^</>+(x,y))^ 

G. =Y,-^s1+\x,y){-icP^{x,y)f, (9.21) 
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where s'^, k = 0,1,..., are as in (8.54\) and 

U{x,y), f.{x,y) e C^iX x X; if (A"''? r (X), A°''? r (X))). 

In the rest of this section, we assume that q = n+. We will compute the 
leading term of iiT^w. For a given point p e X,\et x = {xi,X2,...,X2n-i) be 
the local coordinates as in Theorem |9.2[ We recall that Wo{p) = \/2dx2n-i, 
xip) = 0, Ai.o TpiX) © Ao-i TpiX) = {X a,- ^; a,- G c}, (^(p) | ^(p)) = 
25j,fc, 7, A; = l,...,2n — 1. We identify with some open set in M^r-i vVe 
represent the Hermitian inner product ( | ) on CT{X) by {u\ v) = {Hu,!!), 
where u,v G CT[X], H is a positive definite Hermitian matrix and (,) is 
given by {s,t) = Y^p^^Sjtj, s = (5i,...,52„-i) e C^"-\ t = [ti,...,t2n-i) e 
£2n~i_ Here we identify Cr(X) with C^"~K Let h{x) denote the determinant 
of H. The induced volume form on X is given by ^/ h{x)dx. We have hip] = 
22"-i. Now, 



iK^MoK^i,-,]{x,y] = 

/• 00 /• 00 r 

I I M ^it(l>+{x,w)+is(l)+{w 



JO JO 

Let s = ta, we get 



'^^s+ix, w, t]s+iw,y,s]-\/ h{w)d d t d s . 



{K^(,)oK^i,^\x,y) 

J» 00 /» 00 r» 
( g/f^(x,y,u.,^7)^^(^^ r)5+(u;,y, ta)t/hiw)dw^d(jdt, 
Jo J 

where cj){x,y, w,(j) = ^+(x, w) + (jcj)+{w,y). It is easy to see that 

lm.(j){x,y,w,a)>Q, du,4i{x,y,w,cr)\^=y=u,={(J -l)(x)o{x). 

Thus, x=3/ = if,cr = l,xis real, are real critical points. 

Now, we will compute the Hessian oi(l)dXx=y = w = p,pis real, (7 = 1. 
We write H^{p) to denote the Hessian oi cj) at x=y = w = p,p is veal, (7 = 1. 

%cj)+y 

HJp] has the following form: HJp]= ^ 



. Since 



0, 0,...,0, 72 

^/2 



A, 

* 



{4>+y^{p) = cooip) = ^dx2n-\, we have U^{p] = 

where A is the linear map 

A : A^'" Tp[X) © A"'! TpCX) ^ A^'" Tp[X) © A^'^ TpCX), 

{Au, V) = (((</)+r;^ + (,^4;)u, i;} , V u, veA'-'TpiX)®A°''TpiX) 
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From I I9.18I I, it follows that 71 has the eigenvalues: 

4i I Ai(/?)| Ai I , . . .Ai\^n-i{p)\Ai\K-i{p)\ , (9.22) 

where Ay(p), ; = I,-- - ,(n - 1), are the eigenvalues of the Levi form Lp. We 
have, 

deti^^^) = 2'"-' ■ ■ ■ \K-iipf . (9.23) 

From the stationary phase formula (see Proposition lA.14l l. we get 

r 00 



where 



in the symbol space S^'q x x [0, oo[; if (A"-'? r*(X), A"-'? T*{X]]l a/x,y ) 



C°°(n X J7;if(AO''7r*(X),A''''?r*(X))), j = 0,1,..., and 0i(x,y) is the corre- 
sponding critical value. Moreover, we have 

_i 

aoip, p) = ^det ^^^^^ j slip, p) o slip, p)\fh{p) 

= 2 \Up)\~' ■ ■ ■ I A„-i(p)|~' n"sl{p, p] o slip, p), (9.24) 
where 5° is as in 1 18. 54L We notice that 

(j),ix,x]=o, ((/)i);(x,x)=((/)+);(x,x), ((/)i);(x,x) =((/)+); (x,x). (9.25) 

From l l9J9l l and l l920l l. it follows that 

(^^Mor^,,,)(x,y) = Fi(-/((/)i(x,y)+/0)r" + Gilog(-;((/)i(x,y)+/0)) 

= F+(-/((/)+(x,y) + /O)r'' + G+log(-i(0+(x,y) + im 

(9.26) 

where 

n-l 

Pi=Y.in - 1 - ky.aji-i(p,f+M';, 


/i, Gi e C^inxfl; ^(AO'-? r*(X), A"'-? r*(X))), F+ and G+ are as in Corollarvl931 
From 1 19.251 1 and I I9.26I I. we see that s°ip,p] = Unip.p). From this and II9.24I I. 
we get 

2\Up)\~' ■ ■ ■ \^n-iip)\'' n"slip, p) o slip, p) = slip, p). (9.27) 
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Let ^APo + |tfF) = [u eA°-^T*(X); (p^ + itTFXx,cuo(x))M =o}, where 
is the subprincipal symbol of id}^^ and F is the fundamental matrix of 

n^^\ From the asymptotic expansion of n^^\e''i'+s+), we see that s^ip, p)uG 

^piPo + \tiF] for all u e A"'*? T*[X). (See section 5.) Let 

h = i^\^A-\^n-i\r'n"slip,p). 
From I I9.27I I. we see that 

/f = 7i. (9.28) 

Since X^^Wj, = K^m and 4>+{x,y) = -(j)^iy,x), we have {slf{p,p) = s^ip.p) 
and hence 

11 = h, (9.29) 

where (tt^^)* is the adjoint of tt^', {s1)*{p,p], I* are the adjoints of 5°(p,y9) 
and ii in the space £^{A°-i T*{X],A°-'J T*{X]]] with respect to ( | ) respectively 
Note that dim<y1^(po + |trF) = L (See section 3.) Combining this with 
I l928l l. llHll and sl{p,p) ^ 0, it follows that h ■ A°-'^T*{X) K°''iT*{X) is 
the orthogonal projection onto jYpipl + |tr^)- 

For a given point p &X, let Zi(x), . . . ,Z„_i(x) be an orthonormal frame 
of K^'^TxiX), for which the Levi form is diagonalized at p. Let ej{x), j — 
1, . . . , n - 1 denote the basis of A°'^ T*{X), which is dual to Z,(x), j — l,...,n — 
I. Let Pij{x], j = l,...,n — Ihe the eigenvalues of the Levi form L^. We 
assume that Xjip] > if I < j < n+. Then h = YYjZ"^ ej{p)''ef^'*ip] at p. 
(See section 3.) Summing up, we have proved 

Proposition 9.4. For a given point p & X, let Zi{x),...,Zn-i{x) be an or- 
thonormal frame o/Ai" Tx{X), for which the Levi form is diagonalized at p. 
Let ej[x), j — l,...,n-l denote the basis o/A°'^ T*{X), which is dual toZj{x), 
j — I,... ,n — I. LetXj{x), j = l,...,n — l be the eigenvalues of the Levi form 
Lx- We assume thatq = n+ and thatXj{p) >Oifl<j< n+. Then 

^ i=n+ 
F+ip, p) = {n- 1)!- I Ai(p)| • • • I TT-" Y\ ejipTejipT'*. 

1 The Szego projection on non-orientable CR manifolds 

In this section, (X,A^°r(X)) is a compact connected not necessarily ori- 
entable CR manifold of dimension 2n — I, n > 2. We will use the same 
notations as before. The definition of the Levi form (see Definition l2.3l l de- 
pends on the choices of coq. However, the number of non-zero eigenvalues 
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is independent of tiie choices of coq. Thus, it makes sense to say that the 
Levi-form is non-degenerate. As before, we assume that the Levi form L is 
non- degenerate at each point of X. We have the following 

Lemma 10.1. Let {n-,n+], ri- + n+ = n — I, be the signature of the Levi- 
form L. (The signature of the Levi-form L depends on the choices ofcoo.) If 
n- ^n+ata point of X, then X is orientable. 

Proof. Since X is connected, n_ / n+ at a point of X implies n- / n+ at each 
point of X. Let X = [JUj, where Uj is a local coordinate patch of X. On Uj, 
we can choose an orthonormal frame coij{x],... ,C0n~i,j{x] for A^'°T*[Uj), 
then cJij(x), . . . , aJ„-ij(x) is an orthonormal frame for A°'^ T*{Uj). The [2n — 
2)-form 

(Dj = i"~^(Oij A oJij A ••• A CO„-ij A7o„-ij 

is real and is independent of the choice of the orthonormal frame. There 
is a real 1-form cooj[x) of length one which is orthogonal to A^-°T*[Uj) © 
A°-^T*{Uj). We take coqj so that n_ < n+ on Uj. Since coqj is unique up to 
sign and < n+ on Uj, for all j, we have ojoj{x) = coo^kM on Uj f] Uk, so 
coq is globally defined. The lemma follows. □ 

We only need to consider the case n_ = n+. We recall that if = n+ 
then □^^'^^ has closed range. In view of the proof of Theorem |8.19[ we have 
the following 

Theorem 10.2. Let n+) be the signature of the Levi form. We assume 
thatq — n- — n+. Putt. — {(x,^) e T*{X]\0; = AcoqCx), A 7^ 0}, where coq is 
the locally unique real 1 form determined up to sign by ||cooll = 

co^LiA^-^ T*[X) © Ai'° T*[X)). 

Then n^^^ : Hl^{_X; A°''i T*iX)) H^^^X; A"''? T*iX)] is continuous, for all s e 
R andWFXK„u,)) = diag(E x t], where 

WF'(^,(,,) = {(X, T]) e T%X) X r(X); (x, -q) G WF(i:„w,)} . 

HereWP{K„ui]] is the wave front set of Kj^w) in the sense ofHormander [W^. 
For every local coordinate patch U, we fix a coq on U. We define 

E+ = {(X, E,) e riU] \ 0; ^ = Acoo(x), A > 0} , 
E- = {(X, ^) e r{U) \ 0; ? = Awo(x), A < 0} . 
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We have K^ui) — K^u,) + K^{q] onUx U, where Kn+{x,y) satisfies 

J> oo 


with 

s+{x,y,t)eS'^-\Ux Ux]0,oo[; ^{A^-'^r[X),A°-^r{X)]l 

00 

s+{x,y,t)^Yj^i{x,y)t"-^-^ 

j=o 

in the symbol space S"~\UxU x]0,oo[; £^{A°-i r{XlA°-i T*{X))), where 
4(x,y) e C»(C/ X U; if (A°''? r(X), A"-'' r[X))), ; = 0, 1, . . . , 

4)+{x,y)&C^{UxU), Im(/)+(x,y)>0, 

(j)+{x,x) = Q, (j)+{x,y)^Q ifx^y, 
dx4>+7^0, dy4>+^0 where lm(f)+ — 0, 
dx(j)+{x,y)U=y = ojoix), dycj)+[x,y)\jc=y = -coo[x], 
0+(x,y) = -^+(y,x). 
Moreover, 0+(x,y) satisfies i9.1W . Similarly, 

J> oo 
eil>-i--y^'s-{x,y,t]dt 


with 

oo 

S-[x,y,t)-^Yj^L[x,y)t"-^-^ 

in the symbol space S";o\UxU x]0,oo[; S^{A°'fT*[X),A°-iT*{X])], where 
sL{x,y) e C^iU X U; if (A°''? r(X), A"-'? r{X))), j = 0,l,.... 
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Part II 



On the singularities of the Bergman 
projection for (0, q) forms 
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1 Introduction and statement of the main results 



In this paper, we assume that all manifolds are paracompact. (For the pre- 
cise definition, see page 156 of Kelley [24].) Let M be a relatively compact 
open subset with C°° boundary F of a complex manifold M' of dimension 
n with a smooth Hermitian metric ( | ) on its holomorphic tangent bun- 
dle. (See m.lll .) The Hermitian metric induces a Hermitian metric on the 
bundle of (0, q) forms of M' (see the discussion after m.ll l and section 2) 
and a positive density (<iMO(see I I1.3I I). Let □ be the 5-Neumann Laplacian 
on M (see FoUand-Kohn [14J or 1 11.61 1) and let nf''^ denote the restriction to 
(0, q) forms. For p eT, let Lp be the Levi form of F at p (see I I1.9I I or Def- 
inition |2TTJ. Given q, < q < n — I, the Levi form is said to satisfy condi- 
tion Z[q) at p e F if it has at least n - q positive or at least q + 1 negative 
eigenvalues. When condition Z[q) holds at each point of F, Kohn's esti- 
mates give the hypoellipicity with loss of one dervative for the solutions of 
□(<?) u= f. (See HH or Theorem l3.6[ ) The Bergman projection is the orthog- 
onal projection onto the kernel of in the space. When condition Z(t7) 
fails at some point of F, one is interested in the Bergman projection on the 
level of (0, q] forms. When q = and the Levi form is positive definite, the 
existence of the complete asymptotic expansion of the singularities of the 
Bergman projection was obtained by Fefferman [13J on the diagonal and 
subsequently by Boutet de Monvel-Sjostrand (see [llj) in complete gener- 
ality. If q — n — I and the Levi form is negative definite, Hormander [23] 
obtained the corresponding asymptotics for the Bergman projection in the 
distribution sense. We have been influenced by these works. 

We now start to formulate the main results. First, we introduce some 
standard notations. Let n be a C°° manifold equipped with a smooth den- 
sity of integration. We let T{^1) and T*[iY} denote the tangent bundle of 
and the cotangent bundle of respectively. The complexified tangent bun- 
dle of and the complexified cotangent bundle of will be denoted by 
CT{^) and Cr*(Jl) respectively. We write ( ,) to denote the pointwise du- 
ality between Tin) and T*in). We extend ( ,) bilinearly to CTin) x CT*in]. 
Let £■ be a C°° vector bundle over Jl. The fiber of £" at x e will be denoted 
by Ejc. Let F cc be an open set. The spaces of smooth sections of E over 
Y and distribution sections of E over Y will be denoted by C^iY; E) and 
Q}'{Y; E) respectively. Let S'[Y; E) be the subspace of ®'(F; E) of sections 
with compact support in Y. For 5 e M, we let H^[Y; E) denote the Sobolev 
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space of order 5 of sections of E over Y. Put 

H^^^iY; E) = {ue E);ipue H'{Y; E),^if& C^{Y)} 

^^dH^com^Y'E) = H^JY;E)f]nY;E). _ _ _ 

Let F be a vector bundle over M'. Let C~(M; F), ®'(M; F), H'{M; F) 

denote the spaces of restrictions to M of elements in C°°(M'; F), S)'[M'; F] 

and H'iM'; F) respectively. Let C^iM; F] be the subspace of C°°(M; F] of 

sections with compact support in M. 

Let Ai'° T{M') and A°-^ T{M') be the holomorphic tangent bundle of M' 

and the anti-holomorphic tangent boundle of M' respectively. (See 1 12.31 1.) 

In local coordinates z = {z\,..., z„], we represent the Hermitian metric on 

Ai'Or(M')by 

{u\v)= g{u,v), u,veA^'°TiM'l 

n 

g = 2 gj,k{z)dzj ® dzk, (1.1) 
j,k=i 

where gj,k[z) = J^jiz) e C°°, j,k = l,...,n, and [g j ,k[z))" j^^^ is positive 
definite at each point. We extend the Hermitian metric ( | ) to C T{M'] in a 
natural way by requiring ° T[M') to be orthogonal to A°'^ T[M') and satisfy 
[u I v]^iu\vl u,veA^'^TiM'l 

The Hermitian metric ( | ) on CT{M] induces, by duality, a Hermitian 
metric on Cr*(M) that we shall also denote by ( | ). (See iHJ).) For qeN, 
let AO''? T*[M') be the bundle of (0, q) forms of M'. (See Q.) The Hermi- 
tian metric ( | ) on CT*{M') induces a Hermitian metric on A"'*? T*{M') also 
denoted by ( | ). (See (2M .) 

Let r e C°°(M') be a defining function of F such that r is real, r = on F, 
r < on M and dr ^0 near F. From now on, we take a defining function r 
so that ||dr|| = 1 onF. 

The Hermitian metric ( | ) on CT[M'] induces a Hermitian metric ( | ) on 
C r(F). For z e F, we identify C r*(F) with the space 

[ueCT*iM');{u\dr)^0}. (L2) 

We associate to the Hermitian metric X"fc=i Sj,k{z)dZj ® dZk a real (1, 1) 
form ( see page 144 of Kodaira |i25j) co — jX"i:=i Sj.kdzj A dlk- Let 

dM'^ — (L3) 
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be the volume element and let ( | )m be the inner product on the space 

C°°(M; AO'*? T*{M')) defined by 
r r 



if\h] 



M 



{f\h\dM'): 



I M 



CO' 



M 



if I h) — , f,he C°°(M; A°'^ T*{M')). (1.4) 



Similarly, we take [dT] as the induced volume form on F and let ( | )r be the 
inner product on ^^(r; A"-^ T*iM')) defined by 

r 



if I g\ 



if\g)dT, f,geC'^iT;A''^T*iM% 



(1.5) 



Let d : C°°(M'; A"-'? T*iM')) C°°iM'; AP''^+^ T*{M')) be the part of the ex- 
terior differential operator which maps forms of type (0, q) to forms of type 
(0,^ + 1) and we denote by 5^ : C^{M'; h9'i+^T*{M'))^ C^iM'; K^'^tT^M')) 
the formal adjoint of d . That is 

idf\ h]M'^if\d^h]M', 

f e C^iM'; AO''?r*(M')), h e C°°(M'; A«>'?+i r*(MO), where ( | )m' is defined 
by (g I k]M' = /^Xg I k)idM'l g,ke C^iM'; AO'-? r*(M')). We shall also use 
the notation d for the closure in of the d operator, initially defined on 
C~(M; AO''? T*{M')] and F for the Hilbert space adjoint of ^. 

The ^-Neumann Laplacian on iO,q) forms is then the self-adjoint oper- 
ator in the space L\M; K^'i T*{M')) (see chapter I of [14J) 

d'^'^^^T (1.6) 

We notice that 

Domrf^^ = {K gL^CM; IsP''^ T*[M')); k e DomFp|Dom^, 

d ueDom^, 5ueDom5 } (1.7) 
and C°°(M; AO'*? r*(MO)nDomn('?^ is dense in DomD^'?' for the norm 



u sDomQ 



to). 



u + 



du 



+ 



d u 



(See also page 14 of [14]. 



Let df = d df*+d^d : C^iM'; A"-'? r*(M')) ^ C^iM'; A"''? T*iM')) denote 



the complex Laplace-Beltrami operator on (0, q) forms and denote by a 



the principal symbol of . 

Let ^ be the dual vector of dr. That is {u \ — {u,dr), for all u e 
CUM'). Put 

coo^ridr), (1.8) 
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where J* is the complex structure map for the cotangent bundle. (See I I2.2I I .) 

Let Ai'T(r) and K^-^Tiy) be the holomorphic tangent bundle of F and 
the anti-holomorphic tangent bundle of F respectively. (See I I2.10I I.1 The 
Levi form Lp[Z,W], p eT, Z, W e A^'°Tp[T], is the Hermitian quadratic 
form on ° rp(F) defined as follows: 

For any Z, W e A^-° Tp{ri pickZ, W e C^CF; Ai-" r(F)) that satisfy 

~ _ 1 / ^ ^ \ (1.9) 

Z{p) = Z, Wip) = W. Then Ip(Z, W) = — (^[Z, W]{p) , cooip)) . 

The eigenvalues of the Levi form at p e F are the eigenvalues of the Her- 
mitian form Lp with respect to the inner product ( | ) on Ai'Tp(F). If the 
Levi form is non-degenerate at p e F, let [n-, n+), n- + n+ = n-l,he the 
signature of Lp. Then Z{q) holds at p if and only if q^n.-. 
We recall the Hormander symbol spaces 

Definition 1.1. Let m e M. Let LA be an open set in M' x M'. 

S™ ( [/X ] 0, 00 [; if (A"-'? T{M'l A°'^ r (M'))) 

is the space of all a[x,y, t) e C°°([/x]0,oo[; ^(AO''?T*(M'), A"-'? T*[M'))) such 
that for all compact sets KcU and all a e N^" , ^ e N^" , f e N, there is a con- 
stant c> such that d;^df d/ a[x,y, t) < c(l + (x,y, t] e Kx]0,oo[. 
SJ"o is called the space of symbols of order m type (1,0). We write S]^^ — 

n's^o- 

Let S'PoiU f][MxM) x ] 0, oo [; if (AO'"? T* [M'l A"'-? T*{M'))) denote the space 
of restrictions to Uf^[M x M)x]0,oo[ of elements in 

S;^o([/x]0,oo[;^(A°''?r(MO,A''''?r(M'))). 

Let 

aj e S"J([/n(M X M)x]0,(X)[; if (A"'-? T (M'), A°'^ r{M'))), j = 0,1,2,..., 
with rrij \ — oo, j oo. Then there exists 

a G S™°(C/n(M X M)x]0, oo[; if (A"-'? T* (M'), A"-'? T*iM'))) 
such that 

a- ajeS';'_^{Uf]{MxM)x]0,oo[;^iA'-'^r{M%A°'^r[M^^^ 

0<j<k 

for every keN. (See Proposition 1.8 of Grigis-Sjo strand [16J or Hormander 
ESI.) 
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If a and aj have the properties above, we write 

00 

a^^aj in S;"°(t^n*^Af x M) x [0,(X)[; ^(A°''?r (MO,A°'''r(MO)). 

j=o 

Let 

rf'?^ : L\M; A"''? r{M']) KerD^'?' 

be the Bergman projection, i.e. the orthogonal projection onto the kernel 
of Let K^wiz, w) e ®'(M x M; if (A"-'? T* (M'), AO'-? r*(MO)) be the distri- 
bution kernel of n('?\ 

Let X and F be vector bundles over M'. Let 

C, D : C^iM; X] ®'(M; 

with distribution kernels Kdz, wlKoiz, w) e ®'(M x M; J^iX^j, Y^)). We 
write C = D mod C^^UPliM x M)) if K^z, w) = Kd{z, w) + F{z, w), where 
w)\u e ^^([/nC^ ^{Xw, Yz)) and C/ is an open set in M' x M'. 
Given q,0<q<n-l. Put 

= {z er;Z(<7) fails at z}. (1.10) 

If the Levi form is non-degenerate at each point of F, then Tq is a union of 
connected components of F. 

The main result of this work is the following 

Theorem 1.2. LetM be a relatively compact open subset with C°° boundary 
Tofa complex analytic manifold M' of dimension n. We assume that the 
Levi form is non- degenerate at each point of Y. Letq, 0<q <n — l. Suppose 
that Z{q) fails at some point of Y and that Z{q — Y) andZ{q + Y) hold at each 
point of Y. Then 

KMz, w] e C^(M X M\ diag(F^ x F^); ^(A"-'? r (M'), A"--? r (M'))). 
Moreover, in a neighborhood U o/diag(F^ x F^), iCn^jCz, w) satisfies 

J> oo 
e'^^''"'^'b[z,w,t)dt mod C'^{uf](MxM]) (1.11) 


(for the precise meaning of the oscillatory integral e''t'^^''^^^b{z, w, t)dt, 
see Remark ] 1.3\ below) with 

b[z, w, t) e Sl^iUf](M X M)x]0,(X)[; if (A°''? r (M'), A"'-? T*iM'))l 
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biz, w, w]t^ 



in the space SlQiUf](MxM)x]0,oo[; if (AO'?r*(M'),A°''?r*(M'))), 

boiz,z)^0, zeTq, 

where bjiz,w) e C'^iUf^i'M xM); £^{A0'^ riM'),A°-'i T*iM'))), j = 0,1,..., 
and 

(j}iz,w)eC'^iUfYMxM)], Im0>O, (1.12) 
0(z,z) = O, zeTq, (j)iz,w)^0 if iz,w)^ diagir^xTq), (1.13) 
lm(p{z,w)>0 if{z,w)iYxY, (1.14) 
cj){z,w) = -cj)[w,z). (1.15) 

ForpeTq, we have 

(ji,q){z, dz<p{z, w)) vanishes to infinite order at z = p, 
[z, w) is in some neighborhood of{p, p) in M'. (1.16) 

Forz — w,z & Yq, wehave dz(p = -coo - idr, du,4> =oJo - idr. 

Moreover, we have cj)[z,w) — cj)^{z,w) if z,w e Yq, where cj)-{z,w) e 
C'^iYq X Vq) is the phase appearing in the description of the Szegb projec- 
tion in parti (see also Theorem \7.1^ below). More properties of the phase 
cj){z,w] will be given in Theorem ] 1.4 . 

Remarkl.3. Let cj) and ^7(2, if, be as inTheorem[L2l Lety = (yi, . . . ,y2n-i) 
be local coordinates on T and extend yi, . . .,y2n~i to real smooth functions 
in some neighborhood off. We work with local coordinates 

w^{yi,...,y2n-i,r) 

defined on some neighborhood U of p e Tq. Let u e C^{U; hP''^ T*{M')). 
Choose a cut-off function xit) e C°°(]R) so that /(f) = 1 when |f| < 1 and 
/(?) = when |?| >2. Set 

(5,M)(z) = J e'^^^-"'^'biz,w,t)x{£t)uiw]dtdw. 

Since dyCp ^ where Im^ = (see 1 17.211 1), we can integrate by parts in 
y and t and obtain \im,^o[ Bi;U){z) e C°°(M; A°-iT*{M')). This means that 
B = lim^^o B, : C^iM; A"'*? T*[M')) C^iM; A"'-? T*[M')) is continuous. We 
write B{z, w) to denote the distribution kernel of B. Formally, B{z, w) — 
C e'f^^-'"^^b{z,w,t)dt. 

^ 
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From I I1.16I I and Remark 1.6 of part I it follows that 

Theorem 1.4. Under the assumptions of Theorem ] 1.2[ let p & Tq. We choose 
local complex analytic coordinates z — {zi,...,z„), Zj — Xzj-i + ix2j, j = 
l,...,n, vanishing at p such that the metric on A^-° T[M') is X"=i ^^i ® '^'^i 
atp and r[z) = ^/2lmz „ + + 0(|z|^), where Xj, j = l,...,n-l, 

are the eigenvalues ofLp. (This is always possible. See Lemma 3.2 of 1231.) 
We also write w = [wi, Wn], wj — yij-i + iyij, j = l,...,n. Then, we can 
take^lz, w) so that 

2n-l 



(piz, w) = -\/2x2„-i + ^/2y2n-l - iriz)(l + ^ ajXj + -aznXzn) 

2n—l n—l 

- ir{w)[\ + ^ ajyj + -a2„y2„) + |zy - Wj\^ 

n-1 

+^ iXj{ZjWj - ZjWj) + o[\{z, wf^ (1.17) 



da 



,(<?) 



in some neighborhood of [p, p) in M' x M', where aj — \ [p, -cooip) - 
idrip]],j = l,...,2n. 

We have the following corollary of Theorem ll.2l 

Corollary 1.5. Under the assumptions of Theorem ] 1.2\ and let U be a small 
neighborhood of diag(r^ x r^). Then there exist smooth functions F,G e 
C^^UPliM X M)); ^(A0''7r*(M'), A"'^ T*{M'))) such that 

Kni,) = F(-/(0(z, w) + iO))-"-^ + G log(-z(</>(2, w) + iO]]. 

Moreover, we have 



F = Yj<^-jy-bM' w){-icl){z, w)y+f[z, w){(P[z, w)r+\ 
i=o 

G = 2_^—^^ — bn+j+i[z,w){-i(t)[z,w]y mod C°°{U\\{M x M)) (1.18) 



where fiz, w) e C^iUf^iM x M); ^(A<'''? T* (M'), AO--? T*iM'))). 



If we A"'! T*iM'), let w""'* : A°'i+^ T*[M') A°''J T*{M') be the adjoint of 
left exterior multiplication w^ : A°'i T*{M') A^-''+^ T*[M'). That is, 



{w^u\v)^{u\w^'*v), (1.19) 
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for all u e J\P-iT*[M'), v e J\P-'^+^T*{M'). Notice that w"-* depends anti- 
linearly on w. 

Let A" ! r*(r) be the bundle of boundary (0, 1) forms. (See JzTTTi and 

Proposition 1.6. Under the assumptions of Theorem lL^ let p eTq, q = n-. 
Let Ui[z], U„-i[z) be an orthonormal frame o/Ai-" r^(r), zeT, for which 
the Levi form is diagonalized at p. Let ej{z), j = \,...,n — \ denote the basis 
o/A°'ir*(r), z gY, which is dual to Ujiz], j = l,...,n — I. Let Xj{z), j = 
1, . . . , n — 1 be the eigenvalues of the Levi form Lz, z gY. We assume that 
Xj{p) <Oifl<j < n-. Then 

F{p, p) = n\ I Ai(p)| • • • \Xn-i{p)\ ( W e ^{pf e]^\p)) o[dr[,p)T''[dr{p)f, 

(1.20) 

where F is as in Corollary \L5l 

In the rest of this section, we outline the proof of Theorem II. 2[ We as- 
sume that the Levi form is non- degenerate at each point off. We pause and 
recall a general fact of distribution theory. (See Hormander ||22j.) Let E, F 
be vector bundles over C°° manifolds G and H respectively. We take 
smooth densities of integration on G and H respectively. If A : C^(G; E) 
Q)'{H; F) is continuous, we write KA{x,y) orA{x,y) to denote the distribu- 
tion kernel of A. Then the following two statements are equivalent 

(a) A is continuous: S'{G; E) C°°(H; F), 

(b) KAGC^{HxG;^{Ey,F,)). 

If A satisfies (a) or (b), we say that A is smoothing. Let B : CJ^(G; E) 
S>'{H; F). We write A = BifA — Bisa smoothing operator. 

Let Y denote the operator of restriction to the boundary Y. Let us con- 
sider the map 

F^i^ : H\M; A"-'? r(MO) ^ H°(M; A°-i r[M']] © hHY; A"''? T*{M']l 

u^{dfu,ru). (1.21) 

It is well-known that dimKer Ft"?) < oo and Ker F^'?) c C^(M; A"''? T*{M')). Let 

^(q) . H\M; A"-'? r(MO) ^ KerF^'?^ (1.22) 

be the orthogonal projection with respect to ( | )m. Then, 

J^iq) g (jccf^-^ ^ J^. c^^^O,q r (MO, A"'*? r(MO)). (1.23) 
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Put 

af = Df + K'-'>^ (1.24) 

and consider the map 

piq) . H\M; A"-'? r(MO) ^ H%M; A"''? T* (M')) © ^§(1; A"''? T* (M')), 

u^{nfu,ru). (1.25) 

We can check that F^i^i is injective (see section 4). Let 

P : C°°(r; A"-'? r(MO) ^ C°°(M; A°''? r(MO) (1.26) 

be the Poisson operator for nfp which is well-defined since II1.25I I is injec- 
tive. It is well-known that P extends continuously 

P : H'{T; A"''? T* (M')) ^ H'+^M; A"''? r(MO), V 5 G M. 

(See page 29 of Boutet de Monvel [8J.) Let 

P* : <g"(M; A°'^ T* (M')) ^ ©'(F; A"-'? T* (M')) 

be the operator defined by {P*u \ v)r = [u \ Pv)m, u e S'(M; AP-tT*[M')), 
V e C°°(r; A"-'? T*[M')). It is well-known (see page 30 of [8J) that P* is contin- 
uous: P* : L\M; A^''^ T*iM')) H^^iT; A"''? T*iM']] and 

P* : C°°(M; A"-'? T* (M')) ^ ^^(r; A"'"? T (M')). 

We use the inner product [ | ] on H~^[T; A" *? T*{M')) defined as follows: 

[u I u] — {Pu I Pv)m, 

where u, v G H~^{T; A"'*? T*{M')). We consider (^ r]'^'* as an operator 

(^r)'''* : H-^(r; A°'^ r (M')) ^ H'kT; A°''?-i r(MO). 

Note that [d r]^-* is the pointwise adjoint of ^ r with respect to ( | ). Let 

T : H-^T; A°'''T*{M'])^Ker(drf'* (1.27) 

be the orthogonal projection onto Ker(5r)'^* with respect to [ | ]. That is, 
if M e H-^T; AO'^r*(MO), then (drf'*Tu = and [(/ - T)u \ g] = 0, for all 
g e Ker (5 rY'*. In section 4, we will show that T is a classical pseudodiffer- 
ential operator of order with principal symbol 2{drY'*{drY. For q gN, 
let A''''?r*(F) be the bundle of boundary (0,^7) forms. (See ll2J7ll .) If u G 
Coo(p. ^o.q T*[M')), then u e Ker(^r)'^'* if and only if u e C°°(F; A">'? r*(F)). 
Put 

^= Ty'dP : C°°(F; A°''' T*(T]] C°°(F; A^'^^+i r(F)). (1.28) 
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dp is a classical pseudodifferential operator of order one from boundary 
(0, q) forms to boundary (0, q + l) forms. It is easy to see that dp = ^6+lower 
order terms, where db is the tangential Cauchy-Riemann operator. (See HD 
or section 6.) In section 6, we will show that [dpY = 0. Let 

be the formal adjoint of dp with respect to [ | ]. is a classical pseudodif- 
ferential operator of order one from boundary (0, q + l) forms to boundary 
(0, q) forms. In section 6, we will show that dp^ — ydf P. 
Put 

nf =Tp^^+ d^% : c»(r; a"'-? r (D) ^ c»(r; a"''? r (D). 

For simpUcity, we assume that F = Tq, Vq 7^ Tn-i-q. {Tq is given by m.lOI I.) 
We can repeat the method of part I (see section 7) to construct 

A e L-\ (F; A"''? r(F), A"''? r(F)), B e L% (F; A"'"? r(F), A"'"? r(F)) 

2'2 2'2 

such that Adp^ + B = B + dp^A = I, YpB = 0,d^^B = 0, and B = B^ = 
B^, where L™ 1 is the space of pseudodifferential operators of order m type 

2'2 

(|, |) (see Definition [7TII and is the formal adjoint of B with respect to 
[ I ]. Moreover, KBix,y) satisfies KBix,y) = e''l'-^^-y^'b{x,y,t]dt, where 

^ 

(p-[x,y) and b{x,y, t) are as in Theorem l7.15[ In section 8, we will show that 

U^^^ = PBTiP*PT^P* modC^CMxM) 

and PBTiP*P)-^P*iz,w)= ("^ e'1>''^-'"^^biz,w,t)dt mod C°°(MxM), where 
4>{z, w) and b{z, w, t] are as in Theorem |1.2i 

2 Terminology and notations, a review 

In this section, we will review some standard terminology in complex ge- 
ometry. For more details on the subject, see Kodaira [25J. 

Let £■ be a finite dimensional vector space with a complex structure 
/. By definition, a complex structure / is a M-linear map J : E ^ E that 
satisfies P = —I. Let CE be the complexification of E. That is, CE — 
{u + iv; u,v G E}. Any vector in CE can be written / = u + iv where u, 
V G E and any M-linear map between real vector spaces can be extended 
to a C -linear map between the complexifications, simply by putting Tf = 
Tu + iTv. In particular, we can extend / to a C-linear map / : CE —> CE. 
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Clearly, it still holds that P = —I. This implies that we have a decompo- 
sition as a direct sum CE = A^-^E© A^'^E where Ju = iu if u G A^'^E and 
Ju = -iu if u e A^'^ij. 

Let us now return to our original situation where E = Tp{M'), p e M'. 
Given holomorphic coordinates Zj — Xzj-i + ix2j, j = l,...,n,we get a ba- 
sis for Tp{M') • • • . ^£~i> The complex structure / on Tp{M') is 
defined by 

( r 3 . ^ 
J{j- ) = j—,] = l,...,n, 

■ (2.1) 

/(^) = -^- ,j = l,...,n. 

J does not depend on the choice of holomorphic coordinates. 

The complex structure map : T*{M') T*{M'), for the cotangent 
space is defined as the adjoint of /, that is {Ju ,v) = {u ,JW), u e Tp[M'), 
V G T*{M'). We have 

Jf{dx2j-i) = -dx2j, j = l,...,n, ^2 2^ 

J'idx2j) = dx2j-i, j = l,...,n. 

We can now apply our previous discussion of complex structures on real 
vector spaces to Tp{M') and r*(M'). We then get decompositions 

C Tp{M') = A^'° TpiM') © A°'^ TpiM'l 

C7p(M') = A^'" T*iM') © A°'^ T*iM'). (2.3) 

For M e Ai'TpCM'), v G A^'i r*(MO, -i{u,v) = (uj^v) = {Ju,v) = i{u,v). 
Thus, (u,v) = 0. 

For p,qGN, the bundle of ip, q) forms of M' is given by 

pj>,q T*iM') = A^CA^'" TiM')) A A'?(A'''i r (M')). (2.4) 

That is, the fiber of A? '? T*iM') at zeM' is the vector space Ap(A1''' T*iM'))A 
A'7(A0,i r*(MO) of all finite sums of Wi A ■■■ A Wp A Vi A ■■■ A where Wk e 
Ai'Or*(M0, k ^ l,...,p, Vj G AO'ir*(M'), j = l,...,q. Here A denotes the 
wedge product. 

We recall that if (gj.fc) j.<„ ^ smooth positive definite Hermitian ma- 
trix then the (1, 1) tensor form g — XJfc=i Sj.kdzj ® dzk can be viewed as a 
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Hermitian metric ( | ) on CT(M') in the following way: 

^[g[u,v] + giv,u)) = iu I v)=giu,v), u,veA^'°TiM'), 

[u\w] = 0, ueA^-°T{M'], weA°-^TiM'), 
{u I v) = iu\v), u,veA°-^ T[M'l 

We can check that [Ju \ Ju)^{u \ v), u,v e CTiM']. For t,se T{M'), we 
write t = \{u + u), s = \{v + v), u,v & A^'^T[M'). Then, (f | 5)= \{u \ v) + 
^{u I v) = |Re(M I v) is real. Thus, the Hermitian metric g induces a /- 
invariant Riemannian metric ( | ) on T{M'). 

The Hermitian metric ( | ) on Cr(M) induces, by duality, a Hermitian 
metric on C T*{M) that we shall also denote by ( | ) in the following way. For 
a given point z e M', let A be the anti-linear map A : CT^iM'] CT*{M') 
defined by 

{u\v)^{u,Av), u,v&CT^{M'). (2.5) 

Since ( | ) and (,) are real, A maps Tz{M') to T*{M'). A simple computation 
shows that J^A J =A, JA~^J' =A~^. In particular (since A is anti-linear), 

AA^'° T^{M') = A^-° T*{M'l AA°-^ T,[M'] = A°'^ T*[M']. (2.6) 

For CO, fie CT*iM'], we put 

[co\^) = {A-^[i\A-^co]. (2.7) 

We have (w | /i) = if w e A^'° T*{M'), /i e A^'i r*(MO. 
The Hermitian metric ( | ) on Ap '? T*{M') is defined by 

{Wi A-- - AWp AUiA-- - AUq \ tiA-- - A tp AViA-- - AVq) 

= det [{wj I ffc))^^.^^^det((u,- I v,))^^.^^^, 

Uj, Vk e A"'! r (M'), j,k = l,...,q, Wj,tke A''° r{M'l j,k = l,...,p, 

(2.8) 

and we extend the definition to arbitrary {p, q) forms by sesqui-linearity. 

The Hermitian metric ( | ) on C T[M') induces a Hermitian metric ( | ) 
on Cr(r). For p e F, we have rp(r) = {m e Tp[M'); {u,dr) = {u \ ^) = o}, 
where ^ =A~^dr. Here A is as in 1 12.51 1. 

Put ip = Tp{Y)[^jTp{Y), ^; = r;(r)n/^r;(r) and 

y^A^l (2-9) 

or 
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Here we identify Crp(r) with the space 



We have ^p^{ue Tp{r); {u , cooip)) ^ o] , ^;^[ue T*{r); {u,Y[p))=oy 
(a>o is given by I I1.8I I.) Note that dim^^p — dim.f.'^* — 2n-2. As before, we 
have C^p = A^'" Tp(r) ® A">i TpiT) and CSf * = Ai-" r*(r) © A^-i r*(r), where 



Ju = -iu if ueA°-^T„[r) (2.10) 



Ju = iu if ueA^'"rp(r), 
and 

/^/i = i// if /ieAi'°r(r), 

/'/i = -«/i if /ieA°'^r(r). (2.11) 
We have the orthogonal decompositions with respect to ( | ) 

C TpiT] = A''° TpiT) © A°'i TpCn © {A Y[p); A G C} , 

cr (D = A''° r (D © AO'i r (D © {Xcooipi a e c} . 

We notice that /(« = j(^iJij-]+j-^ = Thus, G 
AO'ir(MO. Nearr, put 

T*-''-' = [ueA''-'T*iM'];iu\driz)) = 0} (2.12) 

and 

T°'' = 1^6 A°'i r,(MO; I (/ F+ -^Xz)) = o| . (2.13) 
We have the orthogonal decompositions with respect to ( | ) 

^0,1 T*f^M') = T*/-^ © {X(dr\zy, A e C} , (2.14) 

AO'i UM') = T^/ © \^X[iY+^)[z); A e c| . (2.15) 

Note that T*-^-^ = A">i r*(r), T^'^ = A"'i r^(r), z e r. 

For i7 e N, the bundle of boundary (0, q) forms is given by 

A''''?r(r)=A'?(A"'ir(n). (2.16) 

Note that 

A°'^r(r) = {ueA°''?r(MO;(u I 5r(z)Ag) = 0, VgeAO'-'-irCM')}. 

(2.17) 

We recall the following 
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Definition 2.1. For p eT, the Levi form Lp(Z, W), Z, W e A^-°Tp{T), is the 
Hermitian quadratic form on ° Tp[r) defined as follows: 

For any Z, W e A^-° Tp{T), pickZ, W e C^CF; Ai-" r(F)) that satisfy 

~ _ 1 / ^ ^ \ (2.18) 

Zip) = Z, Wip) = W. Then Ip(Z, W) = — {[Z, W]ip) , cooip)) . 

Here [Z ,W]^ZW- WZ denotes the commutator ofZ and W. 

It is easy to see that the definition of the Levi form Lp is independent 
of the choices of Z and W. We give it in detail for the convenience of the 
reader (see the discussion before Lemma 2.4 of part I) 

Lemma 2.2. LetZ,W e C^iT; A^-'^TiT)). We have 

^ ([Z ,W]ip),cooip))^-^^ {zip)AWip),d(Oo{p)) . (2.19) 

Definition 2.3. The eigenvalues of the Levi form at p e F are the eigen- 
values of the Hermitian form Lp with respect to the inner product ( | ) on 
A'-'TpiT). 

3 The d -Neumann problem, a review 

In this section, we will give a brief discussion of the d -Neumann problem 
in a setting appropriate for our purpose. General references for this section 
are the books by Hormander lH], HU and Chen-Shaw [6]. 

As in section 1, let M be a relatively compact open subset with smooth 
boundary F of a complex manifold M' of dimension n with a smooth Her- 
mitian metric on its holomorphic tangent bundle. We will use the same no- 
tations as before. We have the following (see page 13 of [14J, for the proof) 

Lemma 3.1. For all f e C~(M; A«''? T*iM')), g e C°°(M; AO-^+i T*iM')), 

(sf I g)M = (/ 1 ^g)M + irf I r(^'-r'*g)r. (3.1) 

where id r)^-* is defined by il.l9f) . We recall that df is the formal adjoint of 
d and y is the operator of restriction to the boundary! . 

We recall that 

Dom^ = {u&L^iM; A"'*? T*iM')); there exist Uj e C~(M; A°''^ r (M')), 
= 1, 2, . . . , and y e L\M; A°-'^+^ T*iM% such that 
Uj ^ M in L\M; A"-'? T*iM')),j oo, and 
'dUj ^vin L\M; A°''?+i r (M')), ; ^ oo}. (3.2) 
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We write du = v. 

The Hilbert space adjoint d of 5 is defined on the domain of d con- 
sisting of all / e U-{M; A° '?+i T*{M')) such that for some constant c > 0, 
(/ I ^g)M < c ||g||, for all g e C^(M; A"'*? T*{M')). For such a /, 

g^{f\dg)M 

extends to a bounded anti-linear functional on L\M\ A"'^ T*{M')) so 

{f\Sg)M = {f\ g)M 

for some / e L2(M; A"'"? T*[M')). We have 5*/ = /. 
From Lemma lHTTl it follows that 

Dom^ Pi C^(M; A°'^ T*{M')) = { m e C~(M; A"-'? r(MO); r(Srf-* w = o} 

(3.3) 

and 

F='^ onDomFpjC^CM; A''''?r(MO). (3.4) 

The 5 -Neumann Laplacian on (0, q) forms is then the operator in the 
space L\M; k^''^T*{M')) 

We notice that □^'^^ is self-adjoint. (See chapter I of HH.) We have 

Domri^^ = {u gL^CM; A°'^r(MO); u eDom^p|Dom^, 
d K eDom^,^M eDom5 }. 

Put Df'?) = DomD^'?) Pi C°°(M ; AO'^ T*{M')). From we have 

D^^^ = { u e C°°(M; A^'-^+i r (M')); ri^rf'* u=0, r(drT'*du = o} . (3.5) 

In view of l IZTTl l, we see that u e D('?^ if and only ifyue C°°(r; A"-'? r*(r)) 
and r^u e C"^{T; A''''?+i r*(r)). We have the following 

Lemma 3.2. Let q > 1. For every u e DomF f] C^CM; A<'''?+i r*(MO), we 
have 

Tue DomF Pi C°°(M; A"'"? r (M')). 

Proof. Let 

w e DomF P C°°(M; A°'''+i r (M')). 
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For g e C~(M; A"''?"! T*[M')l we have 

^iu\'ddg]M-ir(drr-*d*u\rg)r 

^-{r(drr-*'d*u\rg)r. 
Here we used | |3.1| |. Thus, y(drT'*d u = 0. The lemma follows. □ 
Definition 3.3. The boundary conditions 

r(drf-* u=0, r(drf-*'du =0, u e C°°(M , A"-'? T%M')) 
are called d -Neumann boundary conditions. 

Definition 3.4. The d -Neumann problem in M is the problem of finding, 
given a form / G C°°(M; A"-'? T*{M')), another form u e Dt'?) verifying 0^'?^ u = 
f- 

Definition 3.5. Given q, < q < n — I. The Levi form is said to satisfy 
condition Z{q) at e F if it has at least n-q positive or at least q+l negative 
eigenvalues. If the Levi form is non- degenerate at p e F, let n+), ri- + 
n+ = n-l,he the signature of Lp. ThenZ{q) holds at p if and only if q / U-. 

The following classical results are due to Kohn. For the proofs, see |14J . 

Theorem 3.6. We assume thatZ{q) holds at each point of F. Then Ker 
is a finite dimensional subspace ofC^iM; A"''? T*{M']], D^'?^ has closed range 
andU'^'^^ is a smoothing operator. That is, the distribution kernel 

K^iniz, w) e C°°(M X M; ^(A"'-? T (M'), A"'*? T*iM'))). 

Moreover, there exists an continuous operator 

A['^' : L\M; A"''? T* (M')) ^ Domo''?' 

such that + U^'^^ = I on DomD^'?), 

□('?}2vw+n('?^ = / 

on L\M; A"-'? T*{M')). Furthermore, 

A^f'?^(c"(M; A°'^r(M'))) C C°°(M; A"''?r(MO) 

and for each 5 e E and a// / e C°°{M; A"'*? r*(M')), t^ere /s a constant c > 0, 
5Mc/i f/z(2f||Af^^Y||s+i - ^ u>here\\ \\s denotesany of the equivalent norms 
defining H'(M; A°-i T*iM']). 
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Theorem 3.7. Suppose thatZ{q) fails at some point of Y and thatZ{q - 1) 
andZiq + 1) hold at each point ofT. Then, 

n('?^w =(/-Fiv('?-i'F-FN(^+i^^)M, u e DomF Pi c°°(M; A"-'? r (MO), 

(3.6) 

where N^^+^^ andN^'^'^^ areas in Theorem \3M In particular, 
n''?^ : DomF Pi C°°(M; A"-'? T{M')) D^'^l 

4 The operator T 

First, we claim that F^*?' is injective, where F'^^ is given by 1 11 .251 1 . If u e 
KerF(^), then u e C~(M; A"-'? r*(MO), dfu = and yu = 0. We can check 
that 

[nfu I u)M^[dfu I u]m+{K^''^u I u)m 

= (du \'du)M + (d*fU |^u)M + (i^^'^^M I K^''^u)m^O. 

Thus, 1/ e KerF('?^nKerii:('?) (F^'?) is given by (LHJ). We get u = 0. Hence, 
F^*?) is injective. The Poisson operator 

P : C°°(r; A"-'? r (MO) ^ C°°(M; A°'^ r(MO) 

of Qj?^ is well-defined. That is, if u e C°°(r; A"-'? r*(MO), then 

Fm eC°°(M; A°'T(MO), ufPu=Q, rPu = u. 

Moreover, if i; e C°°(M; A<'''?r*(MO) and Lifv 0, then v^Pyv. Further- 
more, it is straight forward to see that 

'dPu=PrdPu, TfPu=Prd*^Pu, u e C^{T; A°-TiM']]. (4.1) 

We recall that (see section 1) P extends continuously 

P : H'iT; A"''? r*(MO) ^ H'+HM; A"''? r*(MO), V 5 G M. 

As in section 1, let P* : (g"(M; A^-'? r*(MO) ^ ©'(F; A"''? r*(MO) be the opera- 
tor defined by 

(F*M I v]r = [u I Pv)m, 

u e S'(M; A">^ r*(MO), V e C°°(F; A"-'? r*(MO). We recall that (see section 1) 
P* is continuous: 

P* : L\M; A"'*? r(MO) ^ H^Y; A°'^ r(MO) 
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and P* : C~(M; A^-^ T*[M')) C^iV; A0>'? T*{M')). 

Let L be a classical pseudodifferential operator on a C°° manifold. From 
now on, we let ctl denote the principal symbol of L. The operator 

P*P : C°°(r; A°''? r (M')) ^ C^CF; A°''? T* (M')) 

is a classical elliptic pseudodifferential operator of order —1 and invertible 
(since P is injective). (See Boutet de Monvel |i7j.) Let Ap be the real Lapla- 
cian on F and let ^/—A^ be the square root of — Ap. It is well-known (see [7J) 
that 

Let (P*P)-i : C^CF; A°''JT*iM')) C°°(F; A°''JT*{M')) be the inverse of P*P. 
{P*P)~^ is a classical elliptic pseudodifferential operator of order 1 with 
scalar principal symbol. We have 

o-f^pi-i =cr,./rT-. (4.3) 



Definition 4.1. The Neumann operator jY^'^'' is the operator on the space 
C°°(r; AO''? T*{M')) defined as follows: 

^i^^f = Y—pf, /eC'^(F;A°''?r(M')). 

The following is well-known (see page 95 of Greiner-Stein ||T5l) 

Lemma 4.2. ^(-7^ : C°°(F; A"'-? T*[M')) C^CF; AO'*? T*iM')) is a classical el- 
liptic pseudodifferential operator of order 1 with scalar principal symbol and 
we have 

a^^Y^„) = cr ^p^. (4.4) 
We use the inner product [ | ] on i/~5(F; A" *? T*[M')) defined as follows: 
[u\v]^{Pu\Pv)m^{P^Pu\v)y, (4.5) 
where u,v & H'^iX; A"'? T*{M')). We consider {d rY'* as an operator 
(drf'* : H-5(r; A°'^ T\M')) H-'kY; A°-i-^ T*{M')). 

Let 

T : H-5(F; A°''? r(M')) ^ Ker (5r)^'* = //-^(F; A°'^ r(r)) (4.6) 

be the orthogonal projection onto Ker(5r)'^'* with respect to [ | ]. That is, 
if w e //-5(F; A<'''?r*(MO), then (drT'*Tu = and [[I - T)u | g] = 0, V g G 
Ker(^r)'^'*. 
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Lemma 4.3. T is a classical pseudodifferential operator of order with prin- 
cipal symbol 2{d rY'*[d rY. Moreover, 

I-T = iP*P)-\'drTR, (4.7) 

where R : C°°(r; AO -? T*{M')) -» C°°(r; A"''?-! T*{M')) is a classical pseudodif- 
ferential operator of order — 1 . 

Proof Let E = 2(^r)^'*((^r)'^'*)^+2((^r)'^'*)^(^r)^'*, 

E : H-5(r; A°''? r (M')) ^ H-5(r; A°'^ r(MO), 
where {{d r)^'*)^ is the formal adjoint of [d r)^-* with respect to [ | ]. That is, 

[(drT'*u \v] = [u\ {(drT'*yv], 
u G H-^(r; AO'^ r*(M')), V G H-^{T; AO'-?"! r*(M')). We can check that 

{(drT'*y= iP'PT\drTiP'P). (4.8) 
Thus, the principal symbol of E is 2{d rY'*{d r)^ + 2{d r]^[d r]^'*. Since 

||dr|| = l = (||^r||' + ||5r||^)^ 

on r, we have 

II- l|2 o 1 

||^r|| =\\dr\f = - on Y. (4.9) 
From this, we can check that 

2(drT-*(drT + 2idrT(drT'* = / : 5 (L; A"''? r (M')) ^ 5 (F; A"''' r (M')), 

where / is the identity map. £ is a classical elliptic pseudodifferential oper- 
ator with principal symbol /. Then dimKerE < oo. Let G be the orthogonal 
projection onto KerE and N be the partial inverse. Then G is a smoothing 
operator and iV is a classical elliptic pseudodifferential operator of order 
with principal sjmibol / (up to some smoothing operator). We have 

EN+G = 2 {(drf'* [(drT'*y+2 iidrf'*) \drT'*^ N + G = I (4.10) 
onH-'2{r;A°'iT*iM')). Put f = 2(^r)^'*((^r)^'*)^iV + G. Note that 

Ker£ = |i/ G H-k^; A°''^ r{M')y, {drf'*u = 0, {(drf-*yu = o} . 
From this and [d rY'* o {d r)^'* = 0, we see that 

Tg G Ker(^r)'''*, g e H'k'^; A"''^ T*{M% 
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From iHUni, we have I-f = 2{{d rY-*]\d r]^-*N and 

[(/- f)g I u] = [2i(d rr-*y(drr'*Ng I u] 
= [2(drT'*Ng\(drT'*u] 
= 0, ue Keridrf'*, g e H-5(r; A"-'? r(MO). 

Thus, g = Tg + {I — T)g is the orthogonal decomposition with respect to 
[ I ]. Hence, T= T. The lemma follows. □ 

Now, we assume that Z{q) fails at some point of T and thatZ(<7 - 1) and 
Z{q + 1) hold at each point of T. Put 

fiiq) ^U^q\l - K^q^l (4.11) 
where K^i^ is as in II1.22I I. It is straight forward to see that 

n(<7) ^ n^'?) _ ^(-7) = (7 _ ^('7))n(^) = p^nc-?) (412) 

and [&i^f = U^i\ (n('?))* = U^i\ where is the formal adjoint of ff'?) 

with respect to ( | )m- 

Proposition 4.4. We assume that Z{q) fails at some point of T and thatZ{q— 
1) andZ{q + 1) hoZd each point of T. Then, 

= (7 - - ^N^'^-^^'d* - d*N^''+^^'d]PnP'Pr^P* u , 

1/ e C°°(M; A"-'? r (MO), (4.13) 

where N^^'^^\ N^i-^^ areas in Theorem UB, T is as in (4.& . In particular, 

n(<7) . c^(M; A°''? T*{M')) D^i\ 

Proof Let v e DomF f] C°°(M; A"-'? r*(M')). From Theorem |3J] and ll4J2l l. 
we see that ff'?) e D('?^ and y = Pt'II^'?) v . Note that 

(/ - p{p*pr'p*f I Prg)M = 0, /, g G c^(M; A"-'? r (MO). 

We have 

(Ui^^PT[P*P)-^P*u I n^'?^i/)M 
= (Pr(P*P)-ip*u |n^'?^i;)M 

= iPTiP*p]-^Pu I P]'n''?'y)M 
= (p(p*p)-^p*u I P]'n^'?^i^)M 

= (M|ff^V)M. (4.14) 
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Thus, 

{u-U'-''^PT[P*PT^P'u |n^'?^i/)M = 0. 

Since Doml* (^C^CM; A^'^i T*{M']] is dense in L\M; A^-''T*[M')), we get 
(u - n('?'Pr(P*P)-ip*M I U^'i'>v)m = 0, for all v e L\M; A"-'? r*(M')). Thus, 

Since 

PT[P*P)-^P* u e DomF Pi C°°(M; A°''? T* (M')), 

from 113 .61 1 and II4.12I I. we get the last identity in II4.13I I. The proposition fol- 
lows. □ 

5 The principal symbols of f^P and f^/p 

First, we compute the principal symbols of d and df . For each point Zq e 
F, we can choose an orthonormal frame ti[z),...,t„-i{z) for T*'°'^ varying 
smoothly with z in a neighborhood of Zq, where r*°'^ is defined by II2.12I I. 
Then (see I I2.14I I) . . . , tn-iiz), := is an orthonormal frame 

forA0.ir*(M0. Let 

Ti{z],...,Tn-i{z),T„[z) 

denote the basis of A°>i Tz[M') which is duEil to ti[z), tn[z). We have (see 
^233) T„ = Note that 

W ~^ drW 

Ti[z), T„-i{z) is an orthonormal frame for A""' T^(F), z e F, (5.1) 

and 

ti{z), t„-i[z) is an orthonormal frame for A'''^ T*{r), z e F. (5.2) 
We have df = ( S J=i Tj ) /, / e C^iM']. If 

fiz)tj, (z) A • • • A tj^^iz) e C^{M'; A'-^ r (MO) 
is a typical term in a general (0, q) form, we have 

n 

j=i 

q _ 
+ YS-lf-'nz)tj,A-A (dtj, )A-Atj^. 
k=l 
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So for the given orthonormal frame we have 

n 

d = ^ t'^ °Tj + lower order terms 



n-l 



+ 



and correspondingly 



dr 



\iY + f\ 

8r\ 



+ lower order terms 



(5.3) 



n-l 



df =2_, ^P* ° V + ^ — ° ~ ^ + '^^^^^ terms. 



dr 



VY+-r\ 

3r\ 



(5.4) 



7=1 

We consider 

ydP : C^{T; A"''? T{M')) C~(r; A°''?+i T{M')) 

and 

yd^P : C°°(r; A''''?+^ T{M')) C°°(r; A"'"? r (M')). 
jdP and 7^/ P are classical pseudodifferential operators of order 1. From 



4.91 1, we know that 



dr 



r. Combining this with 



fj-i 



= ^ on r. We can check that 11/ F+ ^11 = ^/2on 

V 2 II 5 r 1 1 

53J, ll5^ and 1331, we get 



ydP = ^t'}o Tj + [d rf o{iY+ v^-Ar) + lower order terms (5.5) 



and 



n-l 



yd^P = ^ t-'* oT* + {d rf-* o{iY- V-Ar) + lower order terms. (5.6) 

From Lemma lXZl it follows that 

yd^P : C^iT; A°''?+i r (F)) ^ C°°(F; A"'-? r (F)). 

Put 



(5.7) 



E+ = {(x, Awo(x)) e r(F) \ 0; A > 0} , 
S~ = {(x, Acoo(x)) e r (F) \ 0; A < 0} . 



(5.8) 



We recall that coq = J'{d r). In section 8, we need the following 
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Proposition 5.1. The map jid r)^-*dP: C^iT; A"'"? r*(r)) ^ C^iT; A">'? r*(r)) 
is a classical pseudodifferential operator of order one from boundary {0,q) 
forms to boundary (0, q) forms and we have 



rid rT'*dP^ \/-Ar) + lower order terms. (5.9) 

In particular, it is elliptic outside E~. 
Proof Note that 

r(drT'*dP = r(drT'*dP + rdP(drf'* (5.10) 
on the space C^iT; A" *? r*(r)). From 1 15.51 1. we have 

n-l 

r(drT-*dP = Y,[(^rf*tj) o Tj+[(drT-*(drTy[iY+ ^f^) 
+ lower order terms 

and 

n-l 

rdP(drT'*^Y,[^j(^^^'"*] ° Tj+{(drT(drT-*y[iY+ ^f^) 
+ lower order terms. 

Thus, 

n-l 

r(^r)^'*^P + r^P(^r)^'* =^(f^^(^r)^'* + (^r)^'*f;') o Tj 

i=i 

+ [(drndrT'* + (drT'*(drT^ o(iT+ ^f^) 
+ lower order terms. (5.11) 



Note that 

t'>(drT-* + (drT'*t^ = 0, 7 = 1,..., n-l, (5.12) 

and 

(drTidrf-* + (drT-*(drT = ^. (5.13) 

Combining this with I I5.11I I and II5.10I I, we get I I5.9I I. 

Note that a . ^^[x ,?) = -( F,^)+||?|| = ||^||+(wo|^)>0 with equality 
precisely when ^ — — Awo- A > 0. The proposition follows. □ 

For z e r, put 

jo.q = {ue A"'"? r(MO; u = (drfg, g e A°''^-^ r (M')} . (5.14) 
jo.q is orthogonal to A" *? r*(r). In section 7, we need the following 



no 



Proposition 5.2. The operator 

is a classical pseudodifferential operator of order one, 



r{drf'd^P(P'Py^^{iY- -/^A^)/^A^+ lower order terms. (5.15) 
It is elliptic outside S+. 
Proof. Note that 

ridrfd^PiPP)-^ = r(drTd^PiP*Pr' + ydj' PiP* PT\d rf (5.16) 
on the space C°°(r; P'l T*{r)]. From and igJl, we have 



n-l 



rid rf'd^PiP' = ^ ((^ r )^ o [ r o 2 

+ (^(Frf (^rf '*jo(^(/F- ^-Ar)o2v/-Ar) + lower order terms (5.17) 



and 

n-l 



rd^PiP'PrKd rr=Y,{tj-*iSrry[T*o2 ^f^) 

+ ((8rT'*(drTy({iY- -/^)o2/^) + lower order terms. (5.18) 



Thus, 

r(drTd^PiP*Pr' + rd^PiP*Pr\drT 

n-1 

+ ((^r )^'*(^r )^ + (drndrT-*y [{iY- ^f^) o 2 

+ lower order terms. (5.19) 

Combining this with I I5.16I I. 1 15.121 1 and 1 15.131 1, we get 1 15.151 1. The proposition 
follows. □ 

6 The operator d^^ 

Put 

d^= TrdP : C°°(r; A°'^ T*{T)] C°°(r; A^'^^+i T*iT)). (6.1) 
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We recall that (see J4.6I I1 the orthogonal projection T onto Ker[dr]'^-* with 
respect to [ | ] is a classical pseudodifferential operator of order with prin- 
cipal symbol 2[d r)^-*{d r)^ . (See Lemma [Oi l dp is a classical pseudod- 
ifferential operator of order one from boundary (0, q) forms to boundary 
(0,(7 + 1) forms. 

Lemma6.1. We haveidpY = 0. 

Proof. Letu,ve C^iT; A" '? r*(r)). We claim that 

[TydPil - T)rdPu \v] = Q. (6.2) 

We have 

[TxdP{I - T)xdPu I v] = [y'dPil - T)xdPu \ v] (since v e Ker(^r)'^'*) 

= {PrdP{I- T)rdPu I Pv)m 
= (dP{I - T)rdPu \Pv)m (here we used ||4J] |) 
= (P(7 - T]rdPu I d^PvU (since Pv e Dom5*) 
= [(/ - T)ydPu I jdf Pv] (here we used 1 14. 11 1). 

From Lemma |X2l we have jdf Pv e Ker (5 rY'*. Thus, 

[(/ - T)rdPu I r^Pi^] = 0. 

We get lim, and hence TrdPrdPu = TjdPTrdPu, u e C°°(r; A"-'? r*(r)). 
Now, (d^f = TxdPTrdP = TxdPrdP = Tf^'^P = 0. The lemma follows. 

□ 

We pause and recall the tangential Cauchy-Riemann operator. For z e F, 
let n;'^ : A" '? T*{M') A" '? r*(F) be the orthogonal projection map (with re- 
spect to ( I )). We can check that tt"''^ = 2{d r[z)Y-*{d r{z))'^. For an open set 
J7 c F, the tangential Cauchy-Riemann operator: 4 : C°°(C/; A° '?r*(F)) 
C^iU; A0''7+i r*(F)) is now defined as follows: for any e C°°(C/; A">'? r*(F)), 
let U be an open set in M' with C/n F = [/ and pick 0i e C°°(C/; AO*? r*(M')) 
that satisfies 7r^'''(0i(z)) = 0(z), for all z e LA. Then 4.0 is defined to be 
a smooth section in C~([/; A"''?+i r*(F)): z 7r°''^(7'^0i(z)). It is not diffi- 
cult to check that the definition of db is independent of the choice of 0i. 
Since d = 0, we have db = 0. Let db be the formal adjoint of db with re- 
spect to ( I )r, that is (dbf I h]r = if \ 'K'h\, f e C^{U ;AO''?r*(F)), h e 
C°°([/ ; A^ '^+i r*(F)). db is a differential operator of order one from bound- 
ary {0,q + l) forms to boundary (0, q) forms and [db Y = 0. 
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From the definition of db, we know that dt = 2{d r)^'*{d rYjdP. Since the 
principal symbol of T is 2{d rY'*{d r^, it follows that 

dp — dh + lower order terms. (6.3) 

Let 

: C^CF; A''''?+i T* (F)) ^ C°°(F; A°''? T* (F)), (6.4) 

be the formal adjoint of dp with respect to [ | ], that is [dpf \h] = [f \ dp^ h], 
f e C^iV; A°-TiT]), h e C°°(F; A«''?+ir*(F)). d^^ is a classical pseudodif- 
ferential operator of order one from boundary (0, q + l) forms to boundary 
{0,q] forms. 

Lemma 6.2. We have dp = jdf P. 

Proof. Let u e ^^(F; A"'"? r*(F)), v e C°°(F; AO''?+i r*(F)). We have 

[dpu I L*] = [TydPu \ v] — [ydPu \ v] 

= [Py'dPu I Pv]m = (^^M I Pv)m 
= (Pu I 'df*Pv)M — [u I ydf'Pv], 

and the lemma follows. □ 

Remark 6.3. We can check that on boundary (0, q) forms, we have 

^^^ = 7'^P = ^ + lower order terms. (6.5) 

Set 

Uf = 'd^% + Yp'd^^ : ®'(F; r{T)) ^ ®'(F; A"'"^ ^(F)). (6.6) 

□^^ is a classical pseudodifferential operator of order two from boundary 
(0, q) forms to boundary (0, q) forms. We recall that the Kohn Laplacian on 
F is given by uf = 4 ^* + '^'^ : ®'(F; A"-'? r*(F)) ^ @'(F; A"''? r*(F)). From 



J6.3I I and J6.5I I, we see that a^[q) = a^[q) and the characteristic manifold of 

n^p^ is E = E+ljE", where E+, E" are given by 1 15.81 1. (See section 3 of part 
I .) Moreover, a^iq] vanishes to second order on E and we have 

nf = df + L„ (6.7) 

where Li is a classical pseudodifferential operator of order one with 

ctlj = at each point of E. (6.8) 
The following is well-known (see the proof of Lemma 3.1 of part I) 
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Lemma 6.4. E is a symplectic submanifold ofT*{T) if and only if the Levi 
form is non- degenerate at each point of Y. (For the precise definition of sym- 
plectic manifold, see chapterXVlll ofHormanderJJB].) 

Let pp denote the subprincipal symbol of D^^^ (invariantly defined on E) 
and let Ffi(p) denote the fundamental matrix of ct^w) at p e S. We write 

Up 

XrFpip) to denote X l-^il' where ±/Aj are the non- vanishing eigenvalues of 
Fp{p). From I I6.7I I and I I6.8I I, we see that p^ + |tr = + |trFfc on E, where 
PI is the subprincipal symbol of □[f ^ and F^, is the fundamental matrix of 
cr^w). (For the precise meanings of subprincipal symbol and fundamental 
matrix, see section 3 of part I .) We have the following 

Lemma 6.5. Letp ={p,^)e'L. Then 

n—l n—l n—l 

-txFp+p^ =^|Ay||cr,y|+(^Ip(T;,r,)-^ 2f;?^'*Lp(Tfc,ry))cr;y at p, 

(6.9) 

where j = l,...,n — l, are the eigenvalues ofLp and Tj, tj, j = l,...,n — I, 
are as in (5.1\l and i 



Proof. See section 3 of part I . □ 

It is not difficult to see that on S the action of |tf + p^ on boundary 
(0, q) forms has the eigenvalues 

n-l 

^|Aj| |cr,-i.|+^Aj(7iy-^AjCr;F, \J\ = q, 

j=l j^j jeJ (6.10) 

J = Ulj2,---,jq), l<il<i2<---<iq<n-l. 

(See section 3 of part I .) We assume that the Levi form is non- degenerate at 
per. Let (n_, n+), n_ + n+ = n - 1, be the signature of Lp. Since {Y, cuo) = 
-1, we have cr/y > on S+, cj/y < on Tr . 
Let 

\nf{p'p + ]^\iFp){p) = inf |a; A : eigenvalue of (p^ + ^tf F^)(p)| , p e E. 
From 1 16.101 1, we see that at {p, cooip)) e E+, 



At(p,-Wo(p))eE-, 



inf(,.; + it~r F^)j °' "^/"^ . (6.11) 
P 2 I >0, q^n+ 
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< 



Definition 6.6. Given q,0<q<n — l, the Levi form is said to satisfy condi- 
tion Y{q) at p e r if for any |/| = q,J = U1J2, ■ ■ -Jq), l<ji<j2<---<jq< 
n — 1, we have 

n-l 

where Xj, j = l,...,{n — 1), are the eigenvalues of Lp. If the Levi form is 
non- degenerate at p, then the condition is equivalent to q^ n+, n.-, where 
n+), n- + n+ = n — \, is the signature of Lp. 

From now on, we assume that 

Assumption 6.7. The Levi form is non-degenerate at each point of Y. 

By classical works of Boutet de Monvel [9J and Sjostrand [32J, we get the 
following 

Proposition 6.8. is hypoelUptic with loss of one derivative if and only if 
Y{q) holds at each point 0/ F. 

7 The heat equation for D^^^ 

In this section, we will apply some results of Menikoff- Sjostrand |'30] to con- 
struct approximate orthogonal projection for Our presentation is es- 
sentially taken from part I . The reader who is familiar with part I may go 
directly to Theorem l7.15[ 

Until further notice, we work with local coordinates x = (xi, X2, . . . , Xzn-i) 
defined on a connected open set c F. We identify r*(J7) with x 
Thus, the Levi form has constant signature on J7. For any C°° function 
/, we also write / to denote an almost analytic extension. (For the pre- 
cise meaning of almost analytic functions, we refer the reader to Defini- 
tion l.lof Melin-Sjostrand 129].) We let the full symbol of be: 

00 

full symbol of nf ~ ^ qjix, <f ), 

j=o 

where qj[x, ^] is positively homogeneous of order 2- j. 

First, we consider the characteristic equation for dt + n^p\ We look for 
solutions ip[t,x,q]e C°°(E+ x T*[n) \ 0) of the problem 

^-iqo{x,ip'j = 0{\lmipfl VAr>0, 
ip\t=o={x ,q) 
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with lmip{t ,x,ri] > 0. 

Let U be an open set in M" and let f,ge C°°(J7). We write / g if for 
every compact set K c U there is a constant Ck > such that / < CKg, 
g < Ck/ on K. We have the following 

Proposition 7.1. There exists ip[t,x,ri]e C°°(]R+x T*[Q)\0) such thatlmijj > 
u^ff^ equaUty precisely on ({0} x r*(r2) \ 0)|J(E+ x S) awd 5uc/i that (TTTJ 
^oZrfs where the error term is uniform on every set of the form [0, T\x K with 
T>0 and K c T*{U) \ compact. Furthermore, 

ip{t,x,r]] = {x,ri) onE, d^,ri{ip - {x,r])) = onT., 

ip{t,x,Xri) = Xip{Xt,x,ri), A > 0, 

I I ^kl Ho II 

Imj/)(f,x,r?)5^h LI dist((x,■^-^),E))^ f>0, h>l. (7.2) 

1 + ? m m 

Proposition 7.2. T/zere ex/5f5 a function ip{oo,x,ri) e C°°(r*(f2) \ 0) with a 
uniquely determined Taylor expansion at each point of S such that 

For every compact set K c T*[Sl) \ there is a constant Cjf > such that 
lm.xp[QO,x,q) > cjf m (dist((x, -p- 

dx,n[^p{oo,x,q)- (x,)7)) = 0on E. 

If ?^ & CiT*{il)\0], A > awd A|j: < min|Aj|, where ±i\Xj\ are the non- 
vanishing eigenvalues of the fundamental matrix of then the solution 
ip{t,x, q) o/lEU can be chosen so that for every compact set K c r*(r2) \ 
and all indices a, fi,y, there is a constant Ca,p,j,K such that 



d;^dPdlmt,x,q)-ilj{oo,x,r])) 



< Ca.p.r.Ke'^^''-'^^' on R+ x K. (7.3) 



For the proofs of Proposition EU Proposition 17.21 we refer the reader 
to I30l . 

Definition 7.3. We will say that a e C°°(E+ x T*(SY}) is quasi-homogeneous 
of degree j if a{t,x,?[.q] = Xi a{Xt,x, iq) for all A > 0. 



We consider the problem 

{dt + df)u{t,x) = Q inE+xJ7 



u{Q,x)— v{x) 
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We shall start by making only a formal construction. We look for an approx- 
imate solution of 1 17.41 1 of the form u[t,x) = A{t)vix], 



A[t)v{x) 



'\2n-~l 

J 



,7.5, 



(271) 

where formally 

oo 

a[t,x,r])-^'^ajit,x,r]), 

j=o 

aj{t,x,r]) e C°°(E+ x r*(JI); if (AO''?r*(r),AO''?r*(r))), aj[t,x,r]] is a quasi- 
homogeneous function of degree -j . 

We apply dt+n^p^ formally under the integral in 1 17. 5l l and then introduce 
the asymptotic expansion of □^^(ae"'')- Setting [dt + □^^')(fle"^') ~ and 
regrouping the terms according to the degree of quasi-homogeneity. We 
obtain the transport equations 



(7.6) 



{I \^ 
T[t,x,ri,dt,dx)ao = Oi\lmip\ ), VA/" 
Tit,x, rj, dt,d^)aj + lj[t,x, ri,ao,..., aj-i] = 0(|lm VA/". 

Here 

2n—l o o 

J^^^'^x^^ — + qit,x,r]) 

where 

q{t,x,r)) = qi{x,xp'^) + — > — — 

2i d^jd^k dXjdxk 

and lj[t,x,r]) is a linear differential operator acting on ao,ai,...,aj-i. We 
can repeat the method of part I (see Proposition 5.6 of part I) to get the 
following 

Proposition 7.4. Let{n-, n+), n- + n+ = n-l,be the signature of the Levi 
form on Jl. We can find solutions 

fly(f,x,r/)eC°°(E+x r(J^);if(A°''?r(r),A°'^r(r))), ;=0,1,..., 

of the system UM with ao(0,x,?7) = I, aj[0,x,q) = when j > 0, where 
aj[t,x,ri) is a quasi-homogeneous function of degree — j , such that a j has 
unique Taylor expansions on E. Moreover, we can find 

aj [00, X, q) e C'"( rm if (A°''? r (F), A"-'? r (D)), ; = 0, 1, . . . , 
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where ajioo,x, iq) is a positively homogeneous function of degree —j, f o > 
such that for all indices a, l3,Y,j, every compact set K cQ., there exists oO, 
such that 



dld"dj^{aj{t,x, 17) - ajioo, x, 17)) 



<ce-^of|r7|^ + Uh-i-|^|+r (77) 



onR+x(iKxR^"-'^)[]T^, \r]\ > 1. 
Furthermore, for all j = 0,1,..., 

{all derivatives of aj{oo,x, rj) vanish at E+ if cj^ 
all derivatives of aj(oo,x, rj) vanish at E" if q ^ 

and 

ao(oo,x, J7) 7^0 at each point of I!+ if q = n+ 
ao(oo,x, J7) 7^0 at each point of E" if q = U- 



(7.8) 



(7.9) 



Definition 7.5. Let r{x,r]] be a non-negative real continuous function on 
r*(Jl). We assume that r{x,ri) is positively homogeneous of degree 1, that 
is, r{x,Xq) = Xr[x,q), for A > 1, |)7| > 1. For < q < n — 1 and k e M, we say 
that 

a esf(l+ X nn); if (A°''?r(r),A°''?r(r))) 

if a e ^^(1+ X T*in); if (A''>'?r*(r),A">'?r*(r))) and for all indices a, p, y, 
every compact set c J7 and every f > 0, there exists a constant c> such 
that 

djd^d^a{t,x,q) <ce'f-'f^'''H'?|)(i + |r;|)*^+r-|/J|, xeii:, |r/|>l. 

Remark 7.6. It is easy to see that we have the following properties: 

(a) If a e Sf^ , e Sj.^ then ah e ^, a + be S^^;%. 

(b) If a eSf then ^/^/^^^a eS^'^'^^ 

(c) If flj e S/, 7 = 0,1,2,... and kj \ -00 as j — > 00, then there exists 
a e S^" such that a — ^ e S^" , for all 1/ = 1, 2, . . .. Moreover, if S~°° 
denotes HiteK'^f then a is unique modulo S~°°. 

If a and have the properties of (c), we write 

00 

a ~ ^ a j in the symbol space . 



From Proposition 17.41 and the standard Borel construction, we get the 
following 
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Proposition 7.7. Let{n-, n+), n- + n+ = n - I, be the signature of the Levi 
form on n. We can find solutions 

a,(f,x,r;)eC°°(E+x rmif(A°''?r(r),A°''?r(r))), ;=0,1,..., 

of the system UM with (2o(0,x,?7) = /, aj[f),x,rj) = when j > 0, where 
aj[t,x,ri) is a quasi-homogeneous function of degree— j, such thatforsome 
r>0 as in Deflnition \7.5[ 

ajit,x, ri)-ajioo,x, ri) e S^Cl+x r (n); ^(A"'"? T (F), A"'"? r(X])], ; = 0, 1, . . . , 
where 

aj [00, X, T]] e C^[rm if (A°''? r (F), A"-'? r (D)), = 0, 1, . . . , 

and flj(oo, X, rj) is a positively homogeneous function of degree -j . 
Furthermore, for all j =0,1,..., 

{flj((X),x,)7) = in a conic neighborhood of T.+ , \iq^n+, 
aj{oo,x,rj) = Q in a conic neighborhood of Tr , liq^n-. 

Remark!.^. Let b{t,x,r)) e Sf(M+ x T*[ny, Se{A°'iT*[r],A°'iT*{r))) with r > 
0. Weassumethat fo(f,x,}7) = when \r]\ < 1. Let j e C|^(E2«-i)be equal to 

1 near the origin. Put B,[x,y) = j i^j^ e'^'^^''''-'^^-iy-'^)^b{t,x,q)dt^xi£^)dq. 
For u e C^{_n; A°'i r*(F)), we can show that 



lim( 



B,[x,yMy)dy)eC'^in;A°'T[T]) 



and 



is continuous. Formally, 



B{x,y]-- 



u ^lim( Bi,[x,y)u{y)dy) 



• 00 



'0 



Moreover, B has a unique continuous extension: 

B : <S'{n; A"''? T (F)) ^ A°''? r (F)) 

and B(x,y) e C^Cri x ri\ diag(J^ x n); Se{A^-''T*{T),A°'iT*{T))). For the de- 
tails, we refer the reader to Proposition 6.6 of part I . 
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Remark 7.9. Let ait,x,ri) e S^{R+ x r*(Jl); if (AO''?r*(r),AO'^r*(r))). We as- 
sume a{t,x,ri] = 0, if \ri\ < 1 and 

with r > 0, where a[oo,x,r]) e C'^[T*[n); £e[A°-i T*{T),A°-'i T*{T))). Then we 
can also define 

(7.10) 

as an oscillatory integral by the following formula: 



A{x,y)^ 



Jo 



We notice that (- t\iil)\[t,x, r])a[t,x, r)) + a\[t,x, r])) e +i, r > 0. 
We recall the following 

Definition 7.10. Let k eR. ,iT*in); ^{A^-i T*{Y),A^''^ T*{T))) is the space 
of all a e C°°(r*(n); if (A''''?r*(r),AO '?r*(r))) such that for every compact 
sets Ken and all a e N^""!, p e N^"~^, there is a constant Ca,i5,K > such 
that 



dy[aix,^) 



<c„«j,(l + |?|)^-V+V, 



(x,^) e T*(fi), X e K. 1 is called the space of symbols of order k type 



2'2 

^2' 2'' 



Definition 7.11. Let /c e M. A pseudodifferential operator of order k type 
|) from sections of A" '? r*(r) to sections of A°'^ r*(r) is a continuous lin- 
ear map A : C^{n; A" '? r*(r)) A'^''? r*(r)) such that the distribution 
kernel of A is 



with a e i(r*(J7);if(A0''?r*(r),A0''?r*(r))). We call a{x,Q the symbol of 

2'2 

A. We shall write L\ ,{n; A°-i T*{r),A°-i T*[r)] to denote the space of pseu- 



1 1^ 

Jib ui uiuei AC lype [ 

sections of AO''?r*(n. 



do differential operators of order k type (|, |) from sections of A''-'? r*(r) to 



We recall the following classical proposition of Calderon-Vaillancourt. 
(For a proof, see [V9j.) 
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Proposition 7.12. If A e L\ A"'-? r*(r), A"'-? r*(r)). Then, for every 5 e M, 
A is continuous: A : H'„^p(J7; A"'-? r*(r)) ^ Hl-^^{n; A"-'? r*(r)). 

We have the following 

Proposition 7.13. Leta{t ,x,ri)&Sl(M+^T*{n); if (AO''?r*(r),A''''?r*(r))). We 
assume a{t,x,rj) = Q, if\r]\<l and 

a[t,x,r])- a{oo, x, r]) e (M+ x r (5^); if (A"-'? r (F), A"-'? r (F))) 

u^iY/z r > 0, where a{oo,x,r]) e C^irin); i?(A0''7r*(r),A''''?r*(r))). Let 

ei(^'(^-^-n)-{y.'i))a[oo,x, r])^ d dr] 
be as in fZJOl) . Then A e L^'lin; A°-i r*(r), A"-'? r*(r)) u^/f/z 5ymfoo/ 

2'2 

J- 00 


m s^iH7^*(^^); if (A0''7 r*(n, AO''? r*(r))). 

2'2 

Proo/ See Lemma 6.14 and Lemma 6.16 of part I. □ 
From now on, we write 

to denote the kernel of pseudodifferential operator of order A; - 1 type |) 
from sections of A"-'? r*(r) to sections of A"-'? r*(r). Here a[t,x, 17), a(oo, x, rj] 
are as in Proposition l7.13[ 

The following is essentially well-known (See page 72 of 1301 .) 

Proposition 7.14. LetQ be a properly supported pseudodifferential operator 
on of order k>0 with classical symbol q[x, ^) e C'^{T*{^)). Let 

fo(f,x,r/)eS™(l+ X T{p); ^(A°''?r(r),A°'^r(r))). 

We assume that b{t,x,ri) = when\r]\ < 1 andthat 

b{t,x,ri)- b[oo,x, r)) e S™(1+ x r ^(A°''? T{T), A"-" r (F))) 
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with r>0, where b{oo,x, rj) e C"^{T*[n]; 5e{A°-'i r*(r), A">^ T*iT)]) is a classi- 
cal symbol of order m . Then, 

where c{t,x, 17) e 5^+^(1+ x T*{n); if (A"''? r*(r), A"'^ T*{T))), 



in the symbol space S!^+'"{R+ x T*iny, if (A0''7r*(r),A«''?r*(r))), 
c[t,x,r])- cioo,x,r]]eS''+'^iR+x nn); ^{A"'" riT),A°-'^ r{T))), r>0, 

dit,x,r])es-"^iR+ x T%ny, if(A°''?r(r),A°''?r(r))), 

d{t,x,r]]- dioo,x,r]) eS;°^{R+x T*[ny, Se[A°-i T*{r),A°-i T*{r])), r > 0. Here 

c(oo,x, T]) e c^irm; if (A"'"? rcn, A"'"? r(r))) 

J5 a classical symbol of order k + m, 



cj)2{t,x,y, T]) = [x -y)ip[[t,x, 17) - x, r]] - xp{t,y, 17)). Moreover, put 



WehaveQoB = C. 

As in section 1, we put Yq = {z& Y; Z{q) fails at z} and set 

Yr{q) = {{x, e E"; Z(^7) fails at x} , T.+{q] = {(x, e E+; Z(<7) fails at x} . 

From Proposition 17.71 and Proposition 17.141 we can repeat the method of 
part I to get the following 



cit,x, J?) ~ X! ^'xit'^' ri))iRaiip,D,)b) 



d{oo,x, T]) e s'^iriny, ^(a°''? r (mo, a"-'? r (m'))) 

fFor the precise meaning of S~[^ , see Deflnition \ and 



Ra{ip,D,]b^D"^{e'^^^'''-y'^Mt,y,r] 



y=x 
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Theorem 7.15. We recall that we work with the Assumption lUT^ Given q, 
0<q<n-l. Suppose thatZ[q) fails at some point of T. Then there exist 

A e L~\ [T; A"-'? r (n, A"'-? T*[r)], B_, B+ G L° , (F; A°'^ r(r), A°'^ r (F)) 

2' 2 2 ' 2 

such that 

WF'iKsJ = diag{T.'iq) x T~iq)), 

WF'iKsJ = diag(E+(« - 1 - ^7) x E+(« -1-^7)) (7.12) 

and 

Ad^^ + B. + B+ = B. + B+ + d^^A = 1, (7.13) 



dpB. = 0, dp^B. = 0, (7.14) 

^B+ = 0, ^^5+ = 0, (7.15) 

B- = B^_ = B^, (7.16) 

B+ = B\ = Bl, (7.17) 



where B_ and 6+ are the formal adjoints ofB- and B+ with respect to[\] 
respectively and 

WV'[Kb_) = {{X, ?,y, J7) e r(r) X r (D; (X, ?,y, -J7) e WF(^bJ} . 

HereWP{KB_) is the wave front set of Kb in the sense ofHormander /78|/. See 
Deflnition \B.4\ for a review. 

Moreover near diagiTq x T^), i(rB_(x,3/) satisfies 

e'^-^^'^^'Mx,y,Odf 



with 

b{x,y, t)eS'^-\r X rx]0,(X)[; ^{A^''^ T%nA°-'^ r{m, 

00 

b{x,y,t)^Y,^jix,y)t"-'-J in Sj^'Hr x rx]0,oo[; ^(A°''?r(n,A°''?r(r))), 

i=o 

boix,x)^0 ifxeTq, (7.18) 

(A formula for boix,x) will be given in Proposition \ 7.1'A ) whereS^f^, m eM, 
is the Hormander symbol space (see Definition \l.l\) . 

bj{x,y) e ^"(r X r; if (A"-'? r(r), A"-'? r(r))), j = o, i, . . . , 
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and 



(p4x,y]eC°°{TxT), lm(p^{x,y]>0, (7.19) 

(p-ix,x] = 0, ^_(x,y)/0 ifx^y, (7.20) 

dx(j)-~^^, dyCjj^^O where lvs\(j) =Q, (7.21) 

dx4)-{x,y)\x=y = -iOo{x), dy4)-{_x,y)\x=y = iOo{x), (7.22) 

(/)_(x,y) = -^_(y,x). (7.23) 

Similarly, near dia%{T n-i-q x Yn-\-q), 



KB,ix,y)= e'^-^'-y>'cix,y,t)dt 
Jo 

with c{x,y, t)es",-\r x rx]o,oo[; ^(A0''7r;(r),A0''?r*(r))), 

00 

cix,y,t)^^Cjix,y)t"-^- 



-j 

^n—lf 



inS1^\r X rx]0,(X)[; Se{A°-i T*{r),A°-i T*[r))l where 

Cj{x,y] e c^ir X r; if (A"-'? r(r), A"-^ rim, j = o,i,..., 

and—cj)j^lx,y) satifies U.19() - U.23i) . 

We only give the outline of the proof of Theorem l7.15[ For cill the details, 
we refer the reader to section 7 and section 8 of part I . Let 

aj(?,x,r?)eV(^+x ^^^);i^'(A°'''r(^),A°■'?r(^))), 7=0,1,..., 

and aj{oo,x,r]) e C^{T*[ny,Se[A°'iT*[r),A'^''^T*{r))), j =0,1,..., be as in 
Proposition |7.7i We recall that for some r > 

ajit,x, r]]-ajioo,x, r^) e S^Cl+x r (O); ^(A°''? T (F), A"'-? rCH)), ; = 0, 1, . . . . 
Let a{oo,x, rj) ~ ^7=0 ^' S%{T*{rt); if (A"-'? r*(r). A''-'? r*(r))). Let 



!!(r,X,)7)~^flj(f,X,}7) 



<2l 

in S°o{R+ X r*(r?); if (A"-'? r*(r). A"''? r*(r))). We take a{t,x,r]) so that for ev- 
ery compact set c and all indices a, (3, y, k, there exists c > 0, c is 
independent of t, such that 

k 



dld^"d^P{ait,x,ri]-Y,aj{t,x,ri]] 



<cil + \r]\]-''-'+r-\P\^ (7.24) 
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where t e M+, x e i<C, |)7| > 1, and 

a[t,x,ri)- a{oo, x, r]) e S°(M+ x T*[n); ^(A"''? T* (F), A"-'? T* (F))) 
with r > 0. 

Choose / £ Cj^(]R2"~^) so that /(}7) = 1 when |j7| < 1 and xi^) — ^ when 
>2. Set 

gWoo,x,^)-(y,r,))^^Q^^^^^^j^_^^^^^^^j^^_ (7.25) 

Put 

^(x,y)= ^^^|,^_^ J e'W'f°°'^'^'-(^'^)^a(oo,x,r7)dr7. (7.26) 

Since aj{t,x,irj), j — 0,1,..., solve the transport equations 1 17.61 1, we can 
check that B + d^^A = I, B = 0. From the global theory of Fourier inte- 
gral operators (see ||29|), we get Kb = Kb_ + Kb+, wher Kb_ and Kb_^ are as in 
Theorem |7.151 By using a partition of unity we get the global result. 

Remark 7.16. For more properties of the phase 0_(x,y), see Theorem 1.5 
and Remark 1.6 of part I . 

We can repeat the computation of the leading term of the Szego projec- 
tion (see section 9 of part I), to get the following 

Proposition 7.17. Let p eTq, q = ri-. Let Ui[x], Un-i{x) be an orthonor- 
mal frame o/Ai'Tx(F), for which the Levi form is diagonalized at p. Let 
j = l,...,n - I, denote the basis ofA^-^ T*{r), which is dual to Ujix), 
j = l,...,n — l. LetXj{x), j = \,...,n — \,be the eigenvalues of the Levi form 
Ljc. We assume thatXj{p) <0 ifl<j <n-. Then 

, i=n- 

boip, p)^- \Up)\ ■ ■ ■ \K-iip)\ TT-" n ejipTejipT'*, 

where bo is as in i7.18\) . 

In section 8, we need the following 

Proposition 7.18. Suppose thatZ[q] fails at some point of F 
in Theorem l7.15[ Then, 

rdPB- = 0. 



. Let B- be as 
(7.27) 
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Proof. In view of Theorem |7.15i we know that 

Tr'dPB. = d^B- = 0, Yd^PB- = d^^ B- = 0. 
Combining this with yiddf +df d]P = 0,we have 

r'd'fPrdPB- = -rdPyd^PB- = 

and 

rd^Pil - T)rdPB- = rd^PjdPB- - jd^PTxdPB. = 0. (7.28) 

Combining this with Ol, we get xd^P{P*PY^(drYRrdPB- = 0. Thus, 

r(drfdJ'PiP*Pr\'drTRr'dPB^ = 0. 

In view of Proposition l5.2[ we know that 

ridrfdJ'PiP*?)-^ : C°°(r; T* (M')) ^ C°°(r; I°''^T*{M')) 

is elliptic near S~, where T*[M') is as in I I5.14I I. Since 

WF'(i<:Bjcdiag(E~ xS"), 

we get [d r)^RYdPB^ = 0. (See Proposition lB.GI and Proposition lB.71 1 Thus, 
by Ol, (/ - T)ydPB^ = 0. The proposition follows. □ 

8 The Bergman projection 

Given q,0<q<n — I. In this section, we assume that Z[q) fails at some 
point of r and thatZ(i7 — 1) a.ndZ{q + 1) hold at each point of T. In view of 
Propositionim we know that fl^-J^ : C^iM; AP-f T*{M')) D^'f\ Put 

K = rU^''^P : C°°(r; A"'*? T*iT]] C°°(r; A"-'? T*{T)]. (8.1) 

Let be the formal adjoint of K with respect to [ | ] . That is, 

[K^u\v]^[u\Kv], ue ®'(r; A"''^ T*{r)), V e C^{Y; A"''? T\Y)). 
Lemma 8.1. Wehave K^v = Kv , v eC^iV; A9''iT*{Y)). 
Proof. Voxu,v& C°°(r; A"''? T*{Y)), we have 

[Ku I v] = [yn^'^'^Pu I v] 
= &i^Pu I Pv)m 

^{Pu |ff^^Pj^)M 

= [u I Kv]. 

Thus, K^v — Kv. The lemma follows. □ 
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We can extend K to ^'{T; A"-'? r*(r)) Q'{T; A^-i T*{T)) by the following 
formula: [Ku \ u] = [u \ K^v], u e 9'{Y; k^'iT*{T)), v e C°°(r; A°-iT*{r)). 

Lemma 8.2. Let u e Qi'iY; A"'*? r*(r)). We have WF (iCu ) c E" . 

Proof. Let u e ©'(F; A">^r*(r)). We have ir(^rY'*'dP){Ku) = 0. In view of 
Proposition |5.1i we know that yid r]^-*dP is elliptic outside T.~. The lemma 
follows. (See Definition[El) □ 

Lemma 8.3. Let 5_ be as in Theorem \7.15[ We have B^K = KB^ = K. 

Proof. Let A, B„ and B+ be as in Theorem l7.15[ In view of Theorem l7.15[ 
we have 

B. + B+ + Anf = L 

We may replace B+ by / - Ad^^ - B_ and get B_ + B+ +Ad^^ = /. It is easy 
to see that dp^K = 0. Thus, 

K^iB. + B+ + Ad^^]K^{B. + B+)K. (8.2) 

Let u e QJ'{Y; hP-'iT*{Y)). From LemmaO we know that ^{Ku) c Tr . 
Note that WF'CiCs^^) c diag(E+ x S+). Thus, B+Ku e C°°, so B+K is smooth- 
ing (see Lemma [B75] and Proposition IB.7I I and (5_ + B+)K = B-K. From 
this and we get K = B^K and K = = K^b1 = KB-. The lemma 
follows. □ 

We pause and introduce some notations. Let X and Y be C°° vector bun- 
dles over M' and F respectively Let C, D : C~(F; Y] ^'{M; X) with distri- 
bution kernels Kc{z,y), KD{z,y) e @'(M x F; ££{Yy,X^)). We write C = D 
mod C°°(M X F) if Kc{z,y) = KD{z,y) + F{z,y), where F[z,y) e C°°(M x 
T;^iYy,X,)). 

Lemma 8.4. We have 

n('7)pB_ = f[(q)p mod C°°(M X F). (8.3) 

Proof From Lemma [831 we have K = yfl'-'^^P = KB- = yU^'i^pB-. Thus, 
Uiq)p ^ Py&'i^P = PyU^'i^PB- mod C°^(M x F). We get B . □ 

Put 

Q = PB- TiP*P)-^P* : C°°(M; A°'1 T*iM')) C~(M; A°'^ r(MO), (8.4) 
where T is as in 1 14.61 1. 
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Proposition 8.5. We have Q = U^i^ mod C^iM x M). From 032), it follows 
f/zafQ = n('?) modC°°(MxM). 

Proof. We have 

n^Q ^ nWpB_ = n('?)p7^(p*p)-ip* ^od C^CM x M). (8.5) 
Here we used 1 18.31 1. From 1 18. 5l l and the first part of II4.13I I. we get 

n('7)Q = n('/) modC^CMxM). (8.6) 

From Theorem|321 we have n('?)Q = (/ - K^'i'^XI - Tn^^^+^W - FiV('?-i)F)Q, 
where A/^t-J+i) and iV('?-i) are as in Theorem [3S1 From ll727ll . EHl and 
Lemma[62l we see that 'dQ = 0, Tq = mod C°°(M x M). Thus, flt'^^Q = 
(/ - - 5 *Ar('?+i^'^ - 'dN^i-^^T)Q = Q mod C^CM x M). From this and 

I I8.6I I. the proposition follows. □ 

Let X = (xi, . . . , X2„-\) be a system of local coordinates on F and extend 
the functions Xi, . . . , X2n-i to real smooth functions in some neighborhood 
of F. We write (<^i, . . . , E,2n-i, 0) to denote the dual variables of (x, r). We 
write z = (xi, . . . , X2„-i, r), x = (xi, . . .,X2„-i,0], ^ = {^i,..., ^2n-i), C = i^, 0)- 
Until further notice, we work with the local coordinates z — (x, r) defined 
on some neighborhood of p e F. 

We represent the Riemannian metric on T{M'] by 

2n 

h — ^ hj\jciz)dxj ® dXk, dx2n = dr, 
i,k=i 

where hj^kiz) = hk,j{z), j, k = l,...,n, and ^^^^ is positive defi- 

nite at each point of M'. Put (^hj_ic{z)j_^^. ^^^^ = [hj-'^[z))^^j ^<2„- It is well- 
known (see page 99 of Morrow- Kodaira [31]) that 

1 3^ 2/7 — 1 ^2 

= — f h^".2" [z)— + 2Y h''"-Jiz)—— + Tir)] +\ower order terms, 
^ 2v dr'^ ^ drdxi J 

7=1 

(8.7) 

where 

2n—\ Q2 

nr)=Y,h^-\z)——. (8.8) 

j,ic=i oXjdXk 

Note that r(0) = Ap+lower order terms and 

h^^'^^{x) = \, h^"'^ix) = 0, j = l,...,2n-l. 
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We let the full symbol of B^^' be: 

2 

full symbol of uf = ^ qjiz, Q 
j=o 

where qj{z, Q is a homogeneous polynomial of order 2 - ; in (we recall 
that Lif = df + ^(^^). We have the following 

Proposition 8.6. Let (p^ e C°^(r x T) be as in Theorem \7.1^ Then, in some 
neighborhood U o/diag(rg x r^) in M' x M', there exists a smooth function 
${z,y)e C°°((M xT]f]U) such that 

$ix,y) = 4i-{x,y), lm$>0, 

dz4) ^0, dy(p where Im ^ = 0, 

Im0>O ifr^O, (8.9) 

and qo[z, 0^) vanishes to infinite order onr = 0. We write to denote 
acting in the z variables. We have 

d 



dr{z) 



0U,y)lr=O = -«V-^ArU> (</>-);) (8.10) 



in some neighborhood ofx = y , where Re ^J—o■A^ix,i4>-yJ > 0. 
Proof. From 1 18 .71 1 and I I8.8I I. we have 



2n-l 



q,{z, Q = h'"-'"{z)e' + ^ h'"-J{z)e^j + giz, a 

g(x,?) = -^crAr, (8.11) 

where g{z, E,) is the principal symbol of - 1 T{r). 

We consider the Taylor expansion of qo^z, ^) with respect to r, 

q,{z,Q = -9^--ai,,+Y,Si{x>Qri+Y,sAx,QerL (8.12) 
We introduce the Taylor expansion of (j){z,y) with respect to r, 

oo 

${z,y) = 4)-{x,y) + ^(l)j{x,y)rK 
1 

Let 0i(x,y) = -/y'-crAr(x,(0_)^). Since (0_)^U=y = -Wo(x) is real, we 
choose the branch of y^-crAr(x,(0_)^) so that Re ■yj -cr/^^{x,{4^-)'^) > in 
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some neighborhood of X =j/, r = 0. Put ^i(z,y) = ^_(x,y) + r^i(x,y). We 
have <7o('Z>(0i)^) — Oir). Similarly, we can find 4>2ix,y) so that ^70(2,(^2)2) = 
O(r^), where (j)2{z,y) = (l)-.{x,y) + r(j)i{x,y)+r'^(j)2{x,y). Continuing in this 
way we get the phase ^(z,y) such that ^(x,y) = ^_(x,y) and qoiz, van- 
ishes to infinite order on r = 0. The proposition follows. □ 

Remark 8.7. Let 0(z,y ) be as in Proposition |8.6l and let 

rf(z,y, O e S;"(,(^ X rx]0, oo[; if (A"-'? rW), A°''^ 7;W))) 

with support in some neighborhood of 6.ia.g{Tq x Yq). (For the meaning of 
the space S;"o(M x rx]0, oo[; ^{hP'i T*{M'),AP'i T*{M'))), see DefinitionfTTl) 
Choose a cut-off function /(f) e C°°(R) so that /(f) = 1 when |f| < 1 and 
X[t) = when |f| > 2. For all u e C°°(r; A"''? T*iM')), set 

(D,a)(z) = J J e'^^"'^^'rf(z,y, t)xi£t]uiy)dtdy. 

Since Im0 > and dycf) ^ where Im0 =0, we can integrate by parts 
in y, f and obtain \im,^oiDsU)iz) e C°°(M; A^-i T*iM')). This means that 
D = hm^^o D, : ^^(r; AO'*? T*iM')) C°°(M; A*'-'? r*(MO) is continuous. For- 
mally, 

J> 00 
e'^^''y^'d[z,y,t)dt. 


Proposition 8.8. Let 

J. 00 
e^^-^^-y^'b{x,y,t)dt 


be as in Theorem l7.15l We have 

J> 00 
e'^^^-y^'b{z,y,t)dt modC°°(Mxr) 


with biz, y, t] e S1-\M x rx]0, oo[; S^iA°-i r*(r), A"''? T*{M'])], 

00 

fo(z,y,f)~2fo,(z,y)r"-i-^' 

j=o 

inS";o\M X rx]0,oo[; if (A"''? r*(r). A"''? r*(MO)), it;/iere 

;&/z,y ) e C°^(M X F; if (A"'-? r(F), AO'-? T* (M'))), ; = 0, 1, . . . . 
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Proof. Put b[x,y,t)^'^J^i^bj{x,y)t"~^~^ and formally set 

00 

b{z,y,t)^Y,bj{z,y)r-'-i. 

i=o 

We notice that B-(x,y) e C°^(r x r \ diag(r<y x r^); if (AO'-? r*(r), A"-'? r*(r))). 
For simplicity, we may assume that b{x,y, t] = outside some small neigh- 
borhood of diag(r<^ X r^) X M+. Put Bjf^^Cz,}/, t)e'^') = c{z,y, t)e'^K From 
II7.20I I and II8.9I I. we know that near diag(r^ x F^), 0(z,y) = if and only 
if X = y, r — 0. From this observation, we see that if c[z,y, t) vanishes to 
infinite order on diag(F^ x F^) x M+, we can integrate by parts and obtain 



lim 

J 



e'^'ciz,y,t)x{£t)dt =0 modC~(Mxr), 



where is as in Remark [8771 Thus, we only need to consider the Tay- 
lor expansion of b{z,y, t) on x = y, r — 0. We introduce the asymptotic 
expansion of ntj!\be''i"). Setting ~ and regrouping the terms 

according to the degree of homogeneity. We obtain the transport equations 

I T{z,y,d^]bo[z,y) = 

\ Tiz,y,d^]bjiz,y)+ljiz,y,boiz,yl...,bj-iiz,y]] = 0, ; = 1,2,.... 

(8.13) 

Here 

+ R{z,yl 



So qo ~, o oqo ~, o 



where 

2n 



R{z,y] = qi{z,(^^] + — > 

2i d^jd^k dxjdxk 



and Ij is a linear differential operator acting on bo{z,y), bj-i[z,y). 
We introduce the Taylor expansion of bo[z,y) with respect to r, 

oo 

boiz,y)^bo{x,y)+^bl^{x,y)rj . 
1 

Since |y|r=o = O and $'^\r=o = -//^oxTZI^^, we have ^iz,$'J\r=o / 
in some neighborhood of x = y . Thus, we can find fo^(x,y )r such that 

Tiz,y,d,tboix,y) + blix,y)r)^Oi\r\) 
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in some neighborhood of r = 0, x = y. We can repeat the procedure above 
to find blix,y) such that T{z,y,d,\bo{x,y] + Y,l=iKix,y)r''] = 0{\rf) in 
some neighborhood of r — 0, x — y. Continuing in this way we solve the 
first transport equation to infinite order at r — 0, x = y. 

For the second transport equation, we can repeat the method above to 
solve the second transport equation to infinite order at r = 0, x — y. Con- 
tinuing in this way we solve I I8.13I I to infinite order at r = 0, x = y. 

Put B{z,y) — e''i'^^'y^'b{z,y, t)dt. From the construction above, we 
see that 

ufB = t)modC'°(MxT),xB = B.. (8.14) 
It is well-known (see chapter XX of HH) that there exists 

G : C^(M; A"-'? T{M')) C^iM; A"'*? T*[M']) 

such that 



Gn'-f + Pr = I on C°°(M; A°''?r(MO). (8.15) 



From this and ll8J4l l, we have B = (GD^^ + Py)B = PB- mod C~(M x F). 
The proposition follows. □ 



From Proposition |8.8i we have 

J> oo 
e'^(^-y)'c{z,y,t)dt modC°°(Mxr) 


vnth ciz,y, t]eSlo(M xTx]0,oo[; ^(A">'?r*(M0,A0''7r*(M'))), 

oo 

ciz,y,t)^^Cjiz,y)t"- 



'0 

in the space Slo(M x rx]0, oo[; if (A"''? r*(MO, A"'*? T*[M'))). Let 
C* : C°^(M; A°''? T (M')) ^ ©'(F; A"''? T* (M')) 

be the operator defined by (C*u | v]y = (u | Cv)m, u e C^iM; AP-tT*[M')), 
V e C°°(F; A"-'? r*(MO). The distribution kernel of C* is 

J> 00 
e-''^^'-y^'c*{y,z,t)dt modC°°(FxM) (8.16) 


where 

c*iy,z, t)eSl^[r X Mx]0,oo[; if (A''''?r(MO,A°''?r(MO)), 
(c*(y,z,f)/i I v) = (/i I c{z,y,t)v), (jLeA'^-^TWlveA'-^TW), 



c*[y,z,t)-^Yj^*[y,z)t"- 
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in SloiT X Mx]0,oo[; SeiA^'^i T*[M'),A°'i T*{M'))). The integral ll8J6l l is de- 
fined as follows: Let u e C°°(M; A"''? T*{M')). Set 

(C»(y) = 



e-'^(^-y^'c*iy,z,t)x{st)u{z)dtdz, 

'0 



where j is as in Remark [8771 Since dx(j) 7^ where Im^ =0, we can in- 
tegrate by parts in x and t and obtain lim,^oiC*u\y) e C°°{T; A°'^ r*(M')). 
This means that C* = lim.^o C* : C^CM; A°'1 T*[M')) C°°(r; A"'"? T*{M')) is 
continuous. 

We also write w = {yi,.. .,y2n~i, r). We can repeat the proof of Proposi- 
tionlHUto find (p{z, w) e C°°(M x M) such that 0(z,y) = ${z,y), Im0 > 0, 
Im0 > if (z, u/) X r and qo{w,—(p^) vanishes to infinite order on r = 0. 
Since (p-[x,y) = -cj)_[y,x), we can take (p{z, w] so that cj)[z, w) = —cj){w,z). 
As in the proof of Proposition l8.8[ we can find 

a*{w,z, t) e Sl^^iM X M X [0, oo[; if (A"-'? T* (M'), A°''? T* (M'))), 

00 

a*[w,z, f)~^a*(u;,z)f""^' 

i=o 

in Slo(M xMx]Q,oo[; if (A''>'?r*(M0,A0''7r*(M0)), such that 

a*{y,z,t) = c*{y,z,t) 

and □^^(a*(if , z, f)e-''^(^."')f ) vanishes to infinite order on diag(rg xr(y)xE+. 
From |[8J5l l. we have PC*{w,z) = e-'<i>^^-"''i' a*{w,z,t)dt mod C°°(M x 
M). Thus, 

J» 00 
e''t'^^'"'^'a[z,w,t)dt modC°°(MxM), 


a{z, w, t] e Sl^(M x M x [0, oo[; ^(A"-'? r (M'), A"'-? r (M'))), 
(z, If, f) ~^aj(z, u;)f' 



a 



in the space SloiMxMx]0,oo[; ^(A"'"? r*(M'), A"'"? r*(M'))). Note that CP* = 
PB-T{P*P]~^P*. From this and Proposition |8.5[ we get the main result of 
this work 

Theorem 8.9. Given q,0<q<n-l. Suppose that Z[q) fails at some point 
of r and thatZiq - 1) andZ{q + 1) hold at each point of T. Then 

K^i.iz, w) e C^iM X M\ diag(r^ x F^); if (A"-'? r{M'lA'-'^ T*{M'))). 
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Moreover, in a neighborhood U ofdiagiTc, x r^), Xnw(z, w) satisfies 



f oo 



em^'^)t^{z,w,t)dt mod C°°(C/p(M x M)) (8.17) 
with a{z, w, eS^'oCt/flCM x M)x]0,oo[; if (AO.-? r*(M'), A'^-'? T*{M'))), 

oo 

a{z, w, ~^aj(z, w)t^~^ 
j=o 

in the space Slo{U[](MxM)x]0,oo[; ^(AO''?r*(MO,AO''?r;(MO)), 

where ajiz, w) e C^iUfXM x M); ^(A"-'? r*(M'), A*'''? r*(M'))), ; = 0, 1, . . 
and 

0(z,M^)eC°°(C/p|(MxM)), Im0>O, (8.18) 
0(z,z) = O, zer^, (j)iz,w)^0 ifiz,w)^diag{rqXTq), (8.19) 
Im0(z,if)>O (z,M^)^rxr, (8.20) 
0(z,u;) = -0(a',z). (8.21) 

For p e r,^, we have a^wiz, dz^iz, w)) vanishes to infinite order at z = p, 
where (z, w) is in some neighborhood of{p,p) in M'. 

Forz = w, zGTq, wehave dz4> = —oJo — idr, du,4> = Wq — idr. 

J '00 

oo 

bix,y,t)^J^bjix,y)t"-'-^ 

and 

J •00 
e'^^^-"'^'aiz,w,t)dt, 


a{z,w,t)^'^J^Qaj{z,w)t"~L Since 

nw = P5_r(p*p)-^p*, 

{P*Py^ = 2-^— Ar+lower order terms and 

T = 2(drf-*(drf + lower order terms, 

we have 

aoix,x) = 2a /^ix,i(j)-yyix,x))boix,x)2(drix)T'*(drix)T, xgY. 
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Since = coo{x] and \\coo\\ = 1 on T, it follows that 

aoix,x) = 4boix,x](drix]T'*(drix)T. (8.22) 
From this and Proposition !?.!?! we get the following 

Proposition 8.10. Under the assumptions of Theorem W.S^, let p e Vq, q — 
u-. Let Ui{z), Un-i[z) be an orthonormal frame o/Ai'T^(r), z eT, for 
which the Levi form is diagonalized at p. Let ejiz), j = l,...,n- I denote 
the basis o/A°'i r*(r), z e T, which is dual to Uj{z), j = l,...,n-l. LetXj{z), 
j — l,...,n - I be the eigenvalues of the Levi form L^, zeT. We assume that 
?ij{p] <Oifl<j < n-. Then 

i=n- 

aoip, p] = \Up)\ ■ ■ ■ I n~"2 [ Y\ ejipfe'^'^p)) o (^r(p)r (^r(p)r . 

(8.23) 
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Appendices 
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A Almost analytic manifolds, functions and vector fields 



We will give a brief discussion of almost analytic manifolds, functions and 
vector fields in a setting appropriate for our purpose. For more details on 
the subject, see 1291 . 

Let c C" be an open set and let / e C^iW). In this section, we will 
use the following notations: df = X"=i M'^'^i ^^'^ ^ / = X"=i JT^^i- 

Definition A.I. Let W dC" he an open set and let (p{z) be a positive con- 
tinuous function on VK. If / e C°°(VK), we say that / is almost analytic with 
respect to the weight function cj) if, given any compact subset KofW and 

N 



df{z) 



< c(p{z} 



any integer N > 0, there is a constant c > such that 
Vz e K. When (p{z) = \lmz\ we simply say that / is almost analytic. 

Definition A.2. Let /i,/2 e C°°(M^) with W, cj) as above. We say that /i and 
fz are equivalent with respect to the weight function if, given any com- 
pact subset KofW and any integer N > 0, there is a constant c > such 
that |(/i - f2){z)\ < C(j){z)^, Vz e K. When (piz) = \lmz\ we simply say that 
they are equivalent and we write /i ~ /2. 

The following proposition is due to Hormander. For a proof, see 129] . 

Proposition A.3. Let W c C" be an open set and let Wf^ = VyplE". If f e 
C°°(Mi) then f has an almost analytic extension, uniquely determined up to 
equivalence. 

Definition A.4. Let U be an open subset of C" and let A be a C°° submani- 
fold of codimension 2kofU. We say that A is an almost analytic manifold 
if for every point Zq of AQE" there exist an open neighborhood V of Zq in 
U and k complex C°° almost analytic functions fi,...,fk defined on V such 
that 

APl V is defined by the equations /i(z) = • • • = fk{z) = 0, 

d f\,...,d fk are linearly independent over C at every point of V. 

Definition A.5. Let Ai and A2 be two closed submanifolds of an open 
set LA c C". We say that Ai and A2 are equivalent (and we write Ai ~ A2) 
if they have the same intersection with M" and the same dimension and if 
for every open set V cc U and AA e M we have dist(z,A2) < Cjv,v|Imz|^, 

Z&V{^kl,CN,V>^- 

It is trivial that ~ is an equivalence relation and that Ai and A2 are tan- 
gential to infinite order in the real points when Ai ~ A2. 
In section 8 of part I , we need the following 
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Proposition A.6. Let W be an open neighborhood of the origin in C"+™. 
Let fj{z,w), gj[z,w], j = l,...,n, be almost analytic functions on W with 

fj[0,0]^0, gj{0,0) = 0,j = l,...,n,det(^^y _ ^ and d g,,... ,d gn 
are linearly independent over C at the origin. Let 

Ai = {{z, w) e W; fi{z, w) = ... = fn{z, w) = 0} 

and A2 = {{z, w) e W; gi{z, w) = ... = gn{z, w) = O}. if Ai coincides to in- 
finite order with A2 at (0,0) we can then find aj^kiz, w] e C°° in a neigh- 
borhood 0/(0,0), i,k — l,...,n, with det (^flj,fc(0,0)jj^_^ ^ so that gj - 
X!t=i '^j.kfk vanishes to infinite order at (0, 0) in C"+'" , for all j . 

Proof. We write z = x -\- iy , w = u + iv, where x, y e E", u, v G R™. 
From the Malgrange preparation theorem (see Theorem 7.57 of |[22]), it 
follows that gj{x,u) = '^l^-^aj,k{x,u)fic[x,u)-\- r[u], 7 = !,...,«, in a real 
neighborhood of (0,0), where aj^k[x,u] e C°° in a real neighborhood of 
(0,0), j,k = l,...,n. Since Ai coincides to infinite order with A2 at (0,0), 
it follows that r{u] vanishes to infinite order at 0. Since /j:(0,0) = 0, A: = 
l,...,n, we have dgj{0,0] = XLi 0)^/^(0,0), ; = l,...,n. Hence 
det {^aj^iciO,0)^. 7^ 0. Let aj^kiz, w] be an almost analytic extension of 
a j, t (x, u) to a complex neighborhood of (0,0), where j , k = l,...,n. Then 
gj[z, i^)-X"=i <^j,kiz, w)fk{z, w) also vanishes to infinite order at (0,0), for 
all;. □ 

Lemma A.7. Let il, Jl^- be open sets in R". Let k : ^ il^ be a diffeomor- 
phism. Let ^IP and be open sets in C" with Jl^ P| R" = n, P| R" = . 
LetK : f]*^ ^ be an almost analytic extension of k. We take if- and 
50 thatK is a diffeomorphism. If A is an almost analytic manifold of DP, 
then k[A) is an almost analytic manifold of Jl^. Moreover, //" k : Jl*^ — > 
is another almost analytic extension of k and k is a diffeomorphism, then 
K'(A) ~ k(A). Furthermore, if Ai ~ A2, then k(Ai) ~ k(A2), where Ai and A2 
are almost analytic manifolds of O"-. 

We shall now generalize the notion of almost analytic manifolds. We 
have the following 

Definition A.8. Let Xhe a n dimensional real paracompact C°° manifold. 
An cdmost analytic manifold A associated to X is given by 

(a) A locally closed set Ar. (Locally closed means that every point of Ar 
has an neighborhood co in X such that A^ H ^ is closed in co.) 
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(b) A covering of Ak by open coordinate patches k-^ : X D ^ fZ^ c M", 
a e / and almost analytic manifolds A^ c fZ^ with A„ P|]R" := A„k = 
/c„(X„P|Ak). Here n'^ c C" is some open set with ^^^flM" = and 
the Aa shall satisfy the following compatibility conditions: If Kpa = 
Kp o : Ka[Xaf]Xp) Kp[Xa(~]Xp) and if Kpa IS an almost analytic 
extension of Kpa, then Kpai^a) and A^ are equivailent near aill points of 

Kp[Xaf]Xaf]At>). 

The Aa are called local representatives of A and we shall say that two almost 
analytic manifolds A, A' associated to X are equivalent (and we write A ~ A') 
if Ar = Ar and if the corresponding local representatives are equiveilent as 
inib). 

Similarly we extend the notion of almost analytic functions and equiva- 
lence of almost analytic functions. 

Definition A.9. Let W be an open subset of C" and let F be a complex C°° 
vector field on W. We say that V is almost analytic if V[f) is almost analytic 
and V{f] ~ for all almost analytic functions / on W. 

We identify C" with We shall denote the real coordinates by Xj, yj, 
i — \,...,n, and the complex coordinates by Zj =Xj + iyj, j = l,...,n. 

Definition A. 10. Let ly be an open subset of C" andletf7 = X"=i 

J:U^M^^' V = X;=iC;U)4 +S;=irf;(2)^' be complex C~ vector 
fields on W, where aj[z),bj[z),Cj[z),dj[z] e C°°[W), j = l,...,n. We say 
that U and V are equivalent if aj[z) — Cj{z) ~ 0, bj{z) — dj{z) ~ for all 7. If 
U and V are equivalent, we write U ^V. 

Clearly U is almost analytic if and only if LA ~ ^A^^'ir> where Uj, 
j = l,...,n, are almost analytic. We have the following easy lemma 

LemmaA.ll. Let W be an open subset of C" and let V be an almost analytic 

vector field on W. Then V{f) ~ (F+ V]{f) for all almost analytic functions 
f. IfU is a real vector field on W and U{f) ~ V{f) for all almost analytic 
functions f, then U^V+V. 

Proposition A. 12. Let W be an open subset of and letT. be a C°° closed 
suhmanifold of M". Let V be an almost analytic vector field on W. We as- 
sumethatV = Q onT.. Let^{t,p) bethe V+V flow. LetU be a real vector field 
on W such that U ^ V +V. Let^[t,p) be the U flow. Then, for every com- 
pact set K c W, N >0, there is a CN,K{t) > 0, such that ^^[t , p) - ^{t , p)^ < 
CN,K{t)dist[p,T.)^,pGK. 
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Proof. We have the following well-known fact: If Z is a smooth vector field 
on an open set J7 c E" withZ(xo) = 0, ^[t,x) = exp(fZ)(x), then <li{t,Xo) — 
xo and 5;'*(f,x)U=xo- « e N", only depend on [dfzXxo), p e N". In our 
situation, we therefore have that $(f,p), ^{t,p] have the same Taylor ex- 
pansion at every point of E. □ 

The following proposition is useful (see section 2 of 1291 ) 

Proposition A. 13. Assume thatf{x, w) is a C°° complex function in a neigh- 
borhood o/(0,0) m]R"+'" and that 

Im/>0, Im/(0,0) = 0, /;(0,0) = 0, det/;^(0,0) 0. (A.1) 

Let f{z, w), z = x^ iy , w e C™, denote an almost analytic extension of 
f to a complex neighborhood of (0,0) and let ziw] denote the solution of 
j^{z{w), w) = Q in a neighborhood of in C™ . Then, 



-{f{z{w),w))- -—f{z,w%=z[w), wis real, (A.2) 
aw aw 

vanishes to infinite order atOE M™ . Moreover, there is a constant c> such 

that near the origin we have 

\m.f{z{w),w)>c\lmz{w)\^, weE" (A.3) 

and 

\m.f{z{w),w)>cmij^mf{x,w)+\dxf{x,w)^^, u;eE", (A.4) 

where is some open set of the origin in W . 

We call f[z{w), w) the corresponding critical value. 

Proof. For a proof of I IA.3I I. see ||23. We only prove I IA.4I I. In view of the proof 
of (see p.l47 of |29|), we see that 

\m.f{z[w),w)>c{ inf \mf[Rez{w)-t\\v[vz{w)\,w) + \imz{w)\^\ 

VfeR'',|f|<l J 

(A.5) 

for w small, where w is real and c is a positive constant. Using the almost 
analyticity, we get by Taylor's formula: 

/;(x, w) = f",{z[w), w)[x - z[w)) + 0[\x - z{wf + \lmz{wf) (A.6) 

for X, w small, where w is real. Since /"^ is invertible near the origin, we 
have that when w e M™ is close to the origin, 

\lmz{w)f >c\f^[Reziw]-t\lmz[w]\,w)\^ 

for all f e M", I f I < 1, where c is a positive constant. From this and IIA.5I I. we 
getEll- □ 
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In the following, we let z = z{w) be the point defined as above. We recall 
the stationary phase formula of Melin and Sjostrand 

Proposition A. 14. Let f{x, w] be as in Proposition lA 731 Then there are 
neighborhoods U and V of the origin in M" and W" respectively and dif- 
ferential operators Cfj in x of order <2j which are C°° functions ofw&V 
such that 

e'^^udx-[det I 27r/ j) ^ 
<Cwf-~-i t>\, (A.7) 

where u e C^{U x V). Here f and u are almost analytic extensions of f 
and u respectively. The function ^det ^-^^^j^j) ^ the branch of the 

square root of ^det ^-^^^^j^ j j which is continuously deformed into 1 
under the homotopy s e [0,1] i~^il - s)f"^iziw],w) + sl eGL{n,C). 

We need the following asymptotic formula (see ||29|) 

Proposition A.15. Let P be a classical pseudodifferential operator on M". 
Let (fix) e ^^(M") satisfylmip > and dtp ^ wherelvsup = 0. Letu{x) e 
C;^(E"). Ifp[x,^) is the full symbol of P, then 

a! a\ ^ 

a 

with asymptotic convergence in S™^(R" x R+) (for the precise meaning of the 
space S™p see page 5 of il6]), where p is an almost analytic extension ofp 
andp{x,y) = ip{y) - ip{x) -{y-x, ^p'^[x)) . 

Definition A. 16. The function Lp{x,d) defined in an open conic set V c 
M" X \ is called a non- degenerate complex phase function if 

(a) dif^Q. 

(b) ^p[x,9) is positively homogeneous of degree 1. 

(c) Put C = {(x, 9) e V; if'gix, 9) = o}. The differentials i = l,...,N, 
are linearly independent over the complex numbers on C. 

(d) Im(/7>0. 
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Let 0(x, 0) be a non-degenerate complex phase function in a conic open 
subset r of R" xRW. Let = [ix,e]er; (j)g[x,e)^o]. By Euler's homo- 
geneity relation, we have 4>[x,9) — 9. cj)g[x, 9] = on C(j, and therefore Im0 
vanishes on C^. So does d{lm(f)), for otherwise there would be a change of 
sign oflm^. Thus, [{x,(p'^ix,9]]; (x, 0) e C^} c E" x E". 

Let 4> be an almost analytic extension of ^ in a conic open set F c C" x 
C^, rP|(]R" X E^) = F. We can choose (p such that (p is homogeneous of 
degree 1. Set$~ = {dg^$,...,d§^$) Sind$~^{ds,$,...,ds„$). Let 

C^ = {ix,9)ef;$'~ix,9] = 0}, (A.9) 
= {(^. ^)); ^) e Cf} . (A. 10) 

A^ is an almost analytic manifold and 

(A^V = n^^" ^ = {^^' '^xt^' (X, 61) e C4 . 

Let ^1 be another almost analytic extension of (p in F. We have A^ ~ A^^ . 
(For the details, see [29J.) Moreover, we have the following 

Proposition A. 17. For every point [xq, ^q) e (A^V <^nd after suitable change 

of local X coordinates, A^ is equivalent to a manifold -x = ^ eC^ in 
some open neighborhood o/(xo, ^o). where h is an almost analytic function 
and lmh>0 on with equality at (^o- 

Definition A. 18. An almost analytic manifold satisfying the conditions of 
Proposition IA.17I at every real point for some real symplectic coordinate 
(x, E,) is called a positive Lagrangean manifold. 

Let ^p and ip be non- degenerate phase functions defined in small conic 
neighborhoods of (xo, 9q) e R" x R~ and (xq, Wq) e R" x R^ respectively We 
assume that ifgixo, 9o) = 0, ip'^ixo, Wq) = and that if'^ixo, 9o) = ip'^{xo, Wo) = 
^0, where the last equation is a definition. Put Aq = (xo, "^o). Let S™(R" x R^) 
denot the space of classical symbols of order m on R" x R^ (see page 35 
of HI], for the precise meaning). We have the following definition 

Definition A. 19. We say that ip and ip are equivalent at Aq for classical sym- 
bols if there is a conic neighborhood A of (xq, 9o) such that for every distri- 
bution = J e'>^^''^)fl(x,0)d0, where aix,9] e S'^^iR" x R~) with support 

in A, there exists fo(x, w] e S^j ^ (R" x R^) with support in a conic neigh- 
borhood of (xo, Wo) such that A- B eC°°, where B = j e''/'(^'"')fo(x, w)dw 
and vise versa. 
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The global theory of Fourier integral operators with complex phase is 
the following 

Proposition A.20. Let(p andip be non-degenerate phase functions defined 
in small conic neighborhoods of[xo, 60) e M" x and (xo, Wq) e R" x 
respectively. We assume that <^g(xo, 60) — 0, ip'^ixo, Wq) = and that 

Then and ip are equivalent at [xq, ^q) for classical symbols if and only if 
K(p and are equivalent in some neighborhood of {xq, E,q), where ^ and ip 
are almost analytic extensions of if and ip respectively andAi^, are as in 

cool) . 



B The wave front set of a distribution, a review 

We will give a brief discussion of wave front set in a setting appropriate for 
our purpose. For more details on the subject, see ||19l, USD and UTe]. Our 
presentation is essentially taken from HI]. For all the proofs of this section, 
we refer the reader to chapter 7 of [16J, chapter VIII of HD and chapter 
XVIII of 122]. 

We will asume the reader is familiar with some basic notions and facts of 
microlocal analysis such as: Hormander symbol spaces, pseudodifferential 
operators. Nevertheless we recall briefly some of this notions. 

Let c E" be an open set. From now on, we write x" = x"^ ■■■ x"" , d" — 
• • • , D« = D«i • • • D«« and |a| = ai + • • • + a„, where x = (xi, . . . , x„), 
Djcj = — idxj ■ We have the following 

Definition B.l. Let m e M. s;^o(^ x is the space of all a e C°°{n x 
such that for all compact sets K c ^ and aill a G N", ^ G N^, there is a 
constant c> such that 

' <ca+\^\r-\p\,{x,^)eKx. 



d"da[x,^] 



S™o is called the space of symbols of order m. We write Sj J = (~]ST,o' S'^, 



'00 




In order to simplify the discussion of composition of some operators, it 
is convenient to introduce the notion of properly supported operators. Let 
C be a closed subset of 7 x Z. We say that C is proper if the two projections 

Uy:{y,z)eC^yeY 
U^:iy,z]eC^zeZ 
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are proper, that is the inverse image of every compact subset of Y and Z 
respectively is compact. 

A continuous linear operator A : CJ^(Z) is said to be properly 

supported if supp iCi c F x Z is proper. If A is properly supported, then 
A is continuous C^(Z) — > 6'{Y^ and A has a unique continuous extension 
C°°(Z)^®'(F). 

Definition B.2. Let /c e M. A pseudodifferential operator of order k type 
(1,0) is a continuous linear map A : C^(r2) — » such that the distribu- 
tion kernel of A is 

with a G Sf o(r*(f2)). We shall write to denote the space of pseudod- 

ifferential operators of order k type (1,0). 

Definition B.3. Let 

a e s;^o(r*(J7)). Then A is said to be elliptic at (xo, ?o) e r*(f^) \Oifab-lG 
S7_J(r*(f2)) in a conic neighborhood of (xq, (^o) for some b e S7_™(r*(r2)). 

From now on, all pseudofifferential operators in this section will be as- 
sumed properly supported. 

Definition B.4. Let u e @'(n), (xq, <^o) e r*(fl) \ 0. We say that u is C°° near 
(j'l^o. <^o) if there exists A e LPi oiP) elliptic at (xq, ^o)> such that Au e C°°(f2). 
We let WF( I/) be the set of points in 7^(f2) \ 0, where u is not C°°. 

Lemma B.5. Let u e @'(fl). Then u e C~(fl) anrf only WF(m) = 0. 

Let G X J^). Put WF'(^) = {(x, ^,y, r]]; (x, ?,y, -ry) e WF(^)}. 

Proposition B.6. Let A = J e'(^-J')«a(x,?)d? e ^-^oC^)' « ^ S™ (r*(n)). 
LefA i/e the smallest closed cone in r*(r^)\0 such thatforany x e C°°(r*(ri)), 
j(x,A(f) = /(x,(^) and J = m some conic neighborhood of A, u/e /zaye 
/a e S7j(r*(JI)). r/zen WP^Ka) = diag(A x A). Moreover, let u G @'(J^). 
Then,WP{Au)(lk{^;WP{u). 

Proposition B.7. LefJT : C^(ri) ^'{9) with distribution kernel K& ^'{^x 
n). We assume thatWP\K) c {(x, ^, x, ^); (x, ^) e r*(r2) \ 0}. T/zen, there is a 
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unique way of defining jXfu for every u e <g"(fi) so that the map u e <g"(fi) 
JifuG is continuous. Moreover, we have 

WF(jru) c{(x, ^); (X, ^,x, ^)GW¥m 
for some {x,^)&WF{u)]. 
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